Update README.md

README.md

@@ -1,9 +1,242 @@
 ---
-tags:
-- model_hub_mixin
-- pytorch_model_hub_mixin
+license: cc-by-nc-sa-4.0
+metrics:
+- accuracy
+- precision
+- recall
+- f1
+- roc_auc
+base_model:
+- facebook/wav2vec2-xls-r-2b 
 ---
 
-This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
-- Library: [More Information Needed]
-- Docs: [More Information Needed]
+# 🔍 **SSL-AntiDeepfake**
+
+The **SSL-AntiDeepfake** project provides a series of powerful self-supervised learning (SSL) models crafted to **detect deepfake speech** with state-of-the-art accuracy.
+
+
+# 🤖 Available Models
+
+All models are released on Hugging Face 🤗 with two variants:
+- **Default**: Trained with data augmentation
+- **NDA** (No Data Augmentation): Trained without data augmentation
+
+| Model                                    | Variants       |
+|-----------------------------------------------|--------------------|
+| XLS-R-2B-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/xls-r-2b-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/xls-r-2b-anti-deepfake-nda)
+| XLS-R-1B-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/xls-r-1b-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/xls-r-1b-anti-deepfake-nda)
+| MMS-1B-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/mms-1b-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/mms-1b-anti-deepfake-nda)
+| MMS-300M-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/mms-300m-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/mms-300m-anti-deepfake-nda)
+| Wav2Vec2-Large-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/wav2vec-large-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/wav2vec-large-anti-deepfake-nda)
+| Wav2Vec2-Small-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/wav2vec-small-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/wav2vec-small-anti-deepfake-nda)
+| Hubert-Extra-Large-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/hubert-xlarge-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/hubert-xlarge-anti-deepfake-nda)
+
+# 🛠️ Training Code & Repository
+Explore training scripts, config files, and evaluation utilities in our GitHub repository:🔗 [SSL-AntiDeepfake GitHub Repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake)
+
+
+# 🚀 **Model Spotlight: XLS-R-2B-AntiDeepfake-NDA**
+
+# 🌟 **Key Features**
+- **Architecture**: Wav2Vec 2.0 - [`facebook/wav2vec2-xls-r-2b`](https://huggingface.co/facebook/wav2vec2-xls-r-2b) 🔗.
+- **Input**: 16kHz sampled speech with arbitrary length 🎙️.
+- **Output**: Binary classification score (<Fake score 🔴 , Real score 🟢>).
+- **Training Dataset**: Totally 18k hours of fake speech and 56k hours of real speech.
+
+# 🏗️ **Architecture**
+- **Front-end Feature Extractor**: XLS-R-2B.
+- **Back-end Classifier**: A fully connected layer.
+
+# ⚙️ **Training Details**
+- **Optimizer**: AdamW with a learning rate of `1e-7`.
+- **Batch Size**: Dynamic Batching, maximum length per batch is set to 50 seconds.
+- **Data Augmentation**: None
+- **Loss Function**: Cross-Entropy Loss.
+- **Evaluation Metrics**: Equal Error Rate (EER), ROC AUC, Accuracy, Precision, Recall, F1 Score.
+
+# 🚀 **Inference with PyTorch**
+
+⚠️ **Important:**  
+To run inference with this model, you need to install a specific version of `fairseq` and make manual code modifications. For detailed instructions, please refer to the installation guide in our [GitHub repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake).
+
+✅ **Easier Alternatives:**
+We recommend these models for plug-and-play inference: 
+- [MMS-300M-AntiDeepfake](https://huggingface.co/nii-yamagishilab/mms-300m-anti-deepfake)  
+- [MMS-1B-AntiDeepfake](https://huggingface.co/nii-yamagishilab/mms-1b-anti-deepfake)
+
+📦 Dependencies after installing fairseq:
+```
+pip install huggingface-hub safetensors soundfile
+(pip install huggingface-hub==0.31.1 safetensors==0.5.3 soundfile==0.13.1)
+```
+
+🚀 Inference:
+```python
+import os
+import urllib.request
+import torch
+import torchaudio
+import fairseq
+from huggingface_hub import PyTorchModelHubMixin
+
+# === Detect all audio files in a specified folder ===
+folder_path = "/path/to/folder/contains/wavs/"
+audio_formats = (".mp3", ".wav", ".flac", ".m4a")
+
+# === Download Fairseq checkpoint if not present ===
+# The downloaded checkpoint is used for building front-end architecture,
+# its weights will be replaced by the model.safetensors file in this repo.
+ssl_path = "xlsr2_2B_1000k.pt"
+ssl_url = "https://dl.fbaipublicfiles.com/fairseq/wav2vec/xlsr2_2B_1000k.pt"
+
+if not os.path.exists(ssl_path):
+    print(f"Downloading checkpoint to {ssl_path}...")
+    urllib.request.urlretrieve(ssl_url, ssl_path)
+    print("Download complete.")
+else:
+    print(f"{ssl_path} already exists. Skipping download.")
+
+# === Wrapper for the SSL model ===
+class SSLModel(torch.nn.Module):
+    def __init__(self):
+        super().__init__()
+        # The downloaded .pt file is used here
+        model, _, _ = fairseq.checkpoint_utils.load_model_ensemble_and_task([ssl_path])
+        self.model = model[0]
+
+    def extract_feat(self, input_data):
+        # If input has shape (B, T, 1), squeeze the last dim
+        if input_data.ndim == 3:
+            input_data = input_data[:, :, 0]
+
+        # Extract features
+        with torch.no_grad():
+            features = self.model(input_data, mask=False, features_only=True)['x']
+        return features
+
+# === Function for reading and pre-processing waveforms ===
+def load_wav_and_preprocess(wav_path, target_sr=16000):
+    # Load audio file
+    wav, sr = torchaudio.load(wav_path)
+    # Convert to mono if stereo
+    wav = wav.mean(dim=0)
+    # Resample to target sampling rate
+    wav = torchaudio.functional.resample(wav, sr, new_freq=target_sr)
+    # Normalize waveform
+    with torch.no_grad():
+        wav = torch.nn.functional.layer_norm(wav, wav.shape)
+    # Add batch dimension and return
+    return wav.unsqueeze(0)
+
+# === The actual deepfake detection model using SSL frontend + FC backend ===
+class DeepfakeDetector(torch.nn.Module, PyTorchModelHubMixin):
+    def __init__(self):
+        super().__init__()
+        self.ssl_orig_output_dim = 1920
+        self.num_classes = 2
+
+        # Frontend: SSL model
+        self.m_ssl = SSLModel()
+
+        # Backend: Pooling + Classification
+        self.adap_pool1d = torch.nn.AdaptiveAvgPool1d(output_size=1)
+        self.proj_fc = torch.nn.Linear(
+            in_features=self.ssl_orig_output_dim,
+            out_features=self.num_classes,
+        )
+
+    def forward(self, wav):
+        emb = self.m_ssl.extract_feat(wav)  # [B, T, D]
+        emb = emb.transpose(1, 2)           # [B, D, T]
+        pooled_emb = self.adap_pool1d(emb)  # [B, D, 1]
+        pooled_emb = pooled_emb.squeeze(-1) # [B, D]
+        logits = self.proj_fc(pooled_emb)   # [B, 2]
+        return logits
+
+# === Load AntiDeepfake model from Hugging Face===
+model = DeepfakeDetector.from_pretrained("nii-yamagishilab/xls-r-2b-anti-deepfake-nda")
+model.eval()
+
+# === Inference on a folder of audio files ===
+results = []
+for root, _, files in os.walk(folder_path):
+    for file in files:
+        if file.lower().endswith(audio_formats):
+            input_path = os.path.join(root, file)
+            with torch.no_grad():
+                wav = load_wav_and_preprocess(input_path)
+                logits = model(wav)
+                probs = torch.nn.functional.softmax(logits, dim=1)
+                results.append((file, probs.cpu().numpy()[0]))
+
+# Sort results alphabetically by filename
+results.sort(key=lambda x: x[0])
+
+# Print formatted results
+print("\n=== Deepfake Detection Results ===")
+for file_name, prob in results:
+    print(f"{file_name}: real prob = {prob[1]:.3f}, fake prob = {prob[0]:.3f}")
+```
+# 📊 **Performance Metrics**
+Results shown below can be reproduced using scripts provided in our [GitHub repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake).
+
+| Test Database        | ROC AUC | Accuracy | Precision | Recall | F1-score | FPR   | FNR   | EER (%) @ Threshold |
+|---------------------|---------|----------|-----------|--------|----------|-------|-------|----------------------|
+| ADD2023             | 0.983   | 0.947    | 0.957     | 0.972  | 0.964    | 0.124 | 0.028 | 6.84 @ 0.9105        |
+| DeepVoice           | 0.997   | 0.917    | 0.597     | 0.991  | 0.746    | 0.094 | 0.009 | 2.63 @ 0.9801        |
+| FakeOrReal          | 0.999   | 0.975    | 0.996     | 0.953  | 0.974    | 0.003 | 0.047 | 1.18 @ 0.1514        |
+| FakeOrReal-norm     | 0.999   | 0.979    | 0.966     | 0.993  | 0.979    | 0.033 | 0.007 | 1.73 @ 0.7434        |
+| In-the-Wild         | 0.999   | 0.985    | 0.993     | 0.983  | 0.988    | 0.011 | 0.017 | 1.31 @ 0.3748        |
+| WildSVDD-TEST_A     | 0.762   | 0.679    | 0.622     | 0.935  | 0.747    | 0.586 | 0.065 | 30.93 @ 0.9925       |
+| WildSVDD-TEST_B     | 0.718   | 0.714    | 0.754     | 0.846  | 0.797    | 0.546 | 0.154 | 36.11 @ 0.9838       |
+
+
+
+# **Training Set**
+Below is a breakdown of the training set used for fine-tuning.
+
+| 📚 Database             | 🌍 Language         | ✅ Genuine (hrs) | ❌ Fake (hrs) |
+|----------------------|------------------|----------------|-------------|
+| AISHELL3             | zh               | 85.62          | 0           |
+| ASVspoof2019-LA      | en               | 11.85          | 97.80       |
+| ASVspoof2021-LA      | en               | 16.40          | 116.10      |
+| ASVspoof2021-DF      | en               | 20.73          | 487.00      |
+| ASVspoof5            | en               | 413.49         | 1808.48     |
+| CFAD                 | zh               | 171.25         | 224.55      |
+| CNCeleb2             | zh               | 1084.34        | 0           |
+| Codecfake            | en, zh           | 129.66         | 808.32      |
+| CodecFake            | en               | 0              | 660.92      |
+| CVoiceFake           | en, fr, de, it, zh| 315.14         | 1561.16     |
+| DECRO                | en, zh           | 35.18          | 102.44      |
+| DFADD                | en               | 41.62          | 66.01       |
+| Diffuse or Confuse   | en               | 0              | 231.66      |
+| DiffSSD              | en               | 0              | 139.73      |
+| DSD                  | en, ja, ko       | 100.98         | 60.23       |
+| FLEURS               | 102 languages    | 1388.97        | 0           |
+| FLEURS-R             | 102 languages    | 0              | 1238.83     |
+| HABLA                | es               | 35.56          | 87.83       |
+| LibriTTS             | en               | 585.83         | 0           |
+| LibriTTS-R           | en               | 0              | 583.15      |
+| LibriTTS-Vocoded     | en               | 0              | 2345.14     |
+| LJSpeech             | en               | 23.92          | 0           |
+| MLADD                | 38 languages     | 0              | 377.96      |
+| MLS                  | 8 languages      | 50558.11       | 0           |
+| SpoofCeleb           | Multilingual     | 173.00         | 1916.20     |
+| VoiceMOS             | en               | 0              | 448.44      |
+| VoxCeleb2            | Multilingual     | 1179.62        | 0           |
+| VoxCeleb2-Vocoded    | Multilingual     | 0              | 4721.46     |
+| WaveFake             | en, ja           | 0              | 198.65      |
+| Train Set            | Over 100 languages| 56370.00       | 18280.00    |
+
+# **Attribution**
+All SSL-AntiDeepfake models were developed by [Yamagishi Lab](https://yamagishilab.jp/) at the National Institute of Informatics (NII), Japan.
+
+All model weights are the intellectual property of NII and are made available for research and educational purposes under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license.
+
+# **Acknowledgments**
+This project is based on results obtained from project JPNP22007, commissioned by the New Energy and Industrial Technology Development Organization (NEDO).
+
+It is also partially supported by the following grants from the Japan Science and Technology Agency (JST):
+- AIP Acceleration Research (Grant No. JPMJCR24U3)
+- CREST (Grant No. JPMJCR20D3)
+- PRESTO (Grant No. JPMJPR23P9)