Update README.md
Browse files
README.md
CHANGED
|
@@ -12,14 +12,30 @@ base_model:
|
|
| 12 |
|
| 13 |
# π **SSL-AntiDeepfake**
|
| 14 |
|
| 15 |
-
The **SSL-AntiDeepfake**
|
| 16 |
-
They are trained to classify speech as either "Real" π’ or "Fake" π΄.
|
| 17 |
-
| Repository | Link |
|
| 18 |
-
|------------|------|
|
| 19 |
-
| SSL-AntiDeepfake | [GitHub Repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake) |
|
| 20 |
|
| 21 |
|
| 22 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
# π **Key Features**
|
| 25 |
- **Architecture**: Wav2Vec 2.0 - [`facebook/wav2vec2-large-960h-lv60-self `](https://huggingface.co/facebook/wav2vec2-large-960h-lv60-self/tree/main) π.
|
|
@@ -43,11 +59,12 @@ They are trained to classify speech as either "Real" π’ or "Fake" π΄.
|
|
| 43 |
β οΈ **Important:**
|
| 44 |
To run inference with this model, you need to install a specific version of `fairseq` and make manual code modifications. For detailed instructions, please refer to the installation guide in our [GitHub repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake).
|
| 45 |
|
| 46 |
-
|
|
|
|
| 47 |
- [MMS-300M-AntiDeepfake](https://huggingface.co/nii-yamagishilab/mms-300m-anti-deepfake)
|
| 48 |
- [MMS-1B-AntiDeepfake](https://huggingface.co/nii-yamagishilab/mms-1b-anti-deepfake)
|
| 49 |
|
| 50 |
-
π¦ Dependencies after
|
| 51 |
```
|
| 52 |
pip install huggingface-hub safetensors soundfile
|
| 53 |
(pip install huggingface-hub==0.31.1 safetensors==0.5.3 soundfile==0.13.1)
|
|
@@ -62,6 +79,9 @@ import torchaudio
|
|
| 62 |
import fairseq
|
| 63 |
from huggingface_hub import PyTorchModelHubMixin
|
| 64 |
|
|
|
|
|
|
|
|
|
|
| 65 |
|
| 66 |
# === Download Fairseq checkpoint if not present ===
|
| 67 |
# The downloaded checkpoint is used for building front-end architecture,
|
|
@@ -138,13 +158,10 @@ model = DeepfakeDetector.from_pretrained("nii-yamagishilab/wav2vec-large-anti-de
|
|
| 138 |
model.eval()
|
| 139 |
|
| 140 |
# === Inference on a folder of audio files ===
|
| 141 |
-
folder_path = "path/to/folder/contains/wavs/"
|
| 142 |
-
supported_formats = (".mp3", ".wav", ".flac", ".m4a")
|
| 143 |
-
|
| 144 |
results = []
|
| 145 |
for root, _, files in os.walk(folder_path):
|
| 146 |
for file in files:
|
| 147 |
-
if file.lower().endswith(
|
| 148 |
input_path = os.path.join(root, file)
|
| 149 |
with torch.no_grad():
|
| 150 |
wav = load_wav_and_preprocess(input_path)
|
|
@@ -161,6 +178,8 @@ for file_name, prob in results:
|
|
| 161 |
print(f"{file_name}: real prob = {prob[1]:.3f}, fake prob = {prob[0]:.3f}")
|
| 162 |
```
|
| 163 |
# π **Performance Metrics**
|
|
|
|
|
|
|
| 164 |
| Test Database | ROC AUC | Accuracy | Precision | Recall | F1-score | FPR | FNR | EER (%) @ Threshold |
|
| 165 |
|----------------------|---------|----------|-----------|--------|----------|-------|-------|----------------------|
|
| 166 |
| ADD2023 | 0.950 | 0.912 | 0.939 | 0.942 | 0.940 | 0.175 | 0.058 | 13.25 @ 0.8520 |
|
|
@@ -171,8 +190,9 @@ for file_name, prob in results:
|
|
| 171 |
| WildSVDD-TEST_A | 0.739 | 0.663 | 0.616 | 0.895 | 0.729 | 0.577 | 0.105 | 34.85 @ 0.9942 |
|
| 172 |
| WildSVDD-TEST_B | 0.654 | 0.696 | 0.740 | 0.836 | 0.785 | 0.583 | 0.164 | 41.67 @ 0.9842 |
|
| 173 |
|
| 174 |
-
|
| 175 |
# **Training Set**
|
|
|
|
|
|
|
| 176 |
| π Database | π Language | β
Genuine (hrs) | β Fake (hrs) |
|
| 177 |
|----------------------|------------------|----------------|-------------|
|
| 178 |
| AISHELL3 | zh | 85.62 | 0 |
|
|
@@ -204,4 +224,17 @@ for file_name, prob in results:
|
|
| 204 |
| VoxCeleb2 | Multilingual | 1179.62 | 0 |
|
| 205 |
| VoxCeleb2-Vocoded | Multilingual | 0 | 4721.46 |
|
| 206 |
| WaveFake | en, ja | 0 | 198.65 |
|
| 207 |
-
| Train Set | Over 100 languages| 56370.00 | 18280.00 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
|
| 13 |
# π **SSL-AntiDeepfake**
|
| 14 |
|
| 15 |
+
The **SSL-AntiDeepfake** project provides a series of powerful self-supervised learning (SSL) models crafted to **detect deepfake speech** with state-of-the-art accuracy.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
|
| 18 |
+
# π€ Available Models
|
| 19 |
+
|
| 20 |
+
All models are released on Hugging Face π€ with two variants:
|
| 21 |
+
- **Default**: Trained with data augmentation
|
| 22 |
+
- **NDA** (No Data Augmentation): Trained without data augmentation
|
| 23 |
+
|
| 24 |
+
| Model | Variants |
|
| 25 |
+
|-----------------------------------------------|--------------------|
|
| 26 |
+
| XLS-R-2B-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/xls-r-2b-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/xls-r-2b-anti-deepfake-nda)
|
| 27 |
+
| XLS-R-1B-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/xls-r-1b-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/xls-r-1b-anti-deepfake-nda)
|
| 28 |
+
| MMS-1B-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/mms-1b-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/mms-1b-anti-deepfake-nda)
|
| 29 |
+
| MMS-300M-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/mms-300m-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/mms-300m-anti-deepfake-nda)
|
| 30 |
+
| Wav2Vec2-Large-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/wav2vec-large-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/wav2vec-large-anti-deepfake-nda)
|
| 31 |
+
| Wav2Vec2-Small-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/wav2vec-small-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/wav2vec-small-anti-deepfake-nda)
|
| 32 |
+
| Hubert-Extra-Large-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/hubert-xlarge-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/hubert-xlarge-anti-deepfake-nda)
|
| 33 |
+
|
| 34 |
+
# π οΈ Training Code & Repository
|
| 35 |
+
Explore training scripts, config files, and evaluation utilities in our GitHub repository:π [SSL-AntiDeepfake GitHub Repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake)
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
# π **Model Spotlight: Wav2Vec2-Large-AntiDeepfake**
|
| 39 |
|
| 40 |
# π **Key Features**
|
| 41 |
- **Architecture**: Wav2Vec 2.0 - [`facebook/wav2vec2-large-960h-lv60-self `](https://huggingface.co/facebook/wav2vec2-large-960h-lv60-self/tree/main) π.
|
|
|
|
| 59 |
β οΈ **Important:**
|
| 60 |
To run inference with this model, you need to install a specific version of `fairseq` and make manual code modifications. For detailed instructions, please refer to the installation guide in our [GitHub repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake).
|
| 61 |
|
| 62 |
+
β
**Easier Alternatives:**
|
| 63 |
+
We recommend these models for plug-and-play inference:
|
| 64 |
- [MMS-300M-AntiDeepfake](https://huggingface.co/nii-yamagishilab/mms-300m-anti-deepfake)
|
| 65 |
- [MMS-1B-AntiDeepfake](https://huggingface.co/nii-yamagishilab/mms-1b-anti-deepfake)
|
| 66 |
|
| 67 |
+
π¦ Dependencies after installing fairseq:
|
| 68 |
```
|
| 69 |
pip install huggingface-hub safetensors soundfile
|
| 70 |
(pip install huggingface-hub==0.31.1 safetensors==0.5.3 soundfile==0.13.1)
|
|
|
|
| 79 |
import fairseq
|
| 80 |
from huggingface_hub import PyTorchModelHubMixin
|
| 81 |
|
| 82 |
+
# === Detect all audio files in a specified folder ===
|
| 83 |
+
folder_path = "/path/to/folder/contains/wavs/"
|
| 84 |
+
audio_formats = (".mp3", ".wav", ".flac", ".m4a")
|
| 85 |
|
| 86 |
# === Download Fairseq checkpoint if not present ===
|
| 87 |
# The downloaded checkpoint is used for building front-end architecture,
|
|
|
|
| 158 |
model.eval()
|
| 159 |
|
| 160 |
# === Inference on a folder of audio files ===
|
|
|
|
|
|
|
|
|
|
| 161 |
results = []
|
| 162 |
for root, _, files in os.walk(folder_path):
|
| 163 |
for file in files:
|
| 164 |
+
if file.lower().endswith(audio_formats):
|
| 165 |
input_path = os.path.join(root, file)
|
| 166 |
with torch.no_grad():
|
| 167 |
wav = load_wav_and_preprocess(input_path)
|
|
|
|
| 178 |
print(f"{file_name}: real prob = {prob[1]:.3f}, fake prob = {prob[0]:.3f}")
|
| 179 |
```
|
| 180 |
# π **Performance Metrics**
|
| 181 |
+
Results shown below can be reproduced using scripts provided in our [GitHub repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake).
|
| 182 |
+
|
| 183 |
| Test Database | ROC AUC | Accuracy | Precision | Recall | F1-score | FPR | FNR | EER (%) @ Threshold |
|
| 184 |
|----------------------|---------|----------|-----------|--------|----------|-------|-------|----------------------|
|
| 185 |
| ADD2023 | 0.950 | 0.912 | 0.939 | 0.942 | 0.940 | 0.175 | 0.058 | 13.25 @ 0.8520 |
|
|
|
|
| 190 |
| WildSVDD-TEST_A | 0.739 | 0.663 | 0.616 | 0.895 | 0.729 | 0.577 | 0.105 | 34.85 @ 0.9942 |
|
| 191 |
| WildSVDD-TEST_B | 0.654 | 0.696 | 0.740 | 0.836 | 0.785 | 0.583 | 0.164 | 41.67 @ 0.9842 |
|
| 192 |
|
|
|
|
| 193 |
# **Training Set**
|
| 194 |
+
Below is a breakdown of the training set used for fine-tuning.
|
| 195 |
+
|
| 196 |
| π Database | π Language | β
Genuine (hrs) | β Fake (hrs) |
|
| 197 |
|----------------------|------------------|----------------|-------------|
|
| 198 |
| AISHELL3 | zh | 85.62 | 0 |
|
|
|
|
| 224 |
| VoxCeleb2 | Multilingual | 1179.62 | 0 |
|
| 225 |
| VoxCeleb2-Vocoded | Multilingual | 0 | 4721.46 |
|
| 226 |
| WaveFake | en, ja | 0 | 198.65 |
|
| 227 |
+
| Train Set | Over 100 languages| 56370.00 | 18280.00 |
|
| 228 |
+
|
| 229 |
+
# **Attribution**
|
| 230 |
+
All SSL-AntiDeepfake models were developed by [Yamagishi Lab](https://yamagishilab.jp/) at the National Institute of Informatics (NII), Japan.
|
| 231 |
+
|
| 232 |
+
All model weights are the intellectual property of NII and are made available for research and educational purposes under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license.
|
| 233 |
+
|
| 234 |
+
# **Acknowledgments**
|
| 235 |
+
This project is based on results obtained from project JPNP22007, commissioned by the New Energy and Industrial Technology Development Organization (NEDO).
|
| 236 |
+
|
| 237 |
+
It is also partially supported by the following grants from the Japan Science and Technology Agency (JST):
|
| 238 |
+
- AIP Acceleration Research (Grant No. JPMJCR24U3)
|
| 239 |
+
- CREST (Grant No. JPMJCR20D3)
|
| 240 |
+
- PRESTO (Grant No. JPMJPR23P9)
|