nii-yamagishilab commited on
Commit
9b00069
Β·
verified Β·
1 Parent(s): 3708ff6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +47 -14
README.md CHANGED
@@ -12,14 +12,30 @@ base_model:
12
 
13
  # πŸ” **SSL-AntiDeepfake**
14
 
15
- The **SSL-AntiDeepfake** models are a series of self-supervised learning (SSL) models fine-tuned on multiple datasets of real and deepfake speech.
16
- They are trained to classify speech as either "Real" 🟒 or "Fake" πŸ”΄.
17
- | Repository | Link |
18
- |------------|------|
19
- | SSL-AntiDeepfake | [GitHub Repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake) |
20
 
21
 
22
- # πŸš€ **Model: Wav2Vec2-Large-AntiDeepfake**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
 
24
  # 🌟 **Key Features**
25
  - **Architecture**: Wav2Vec 2.0 - [`facebook/wav2vec2-large-960h-lv60-self `](https://huggingface.co/facebook/wav2vec2-large-960h-lv60-self/tree/main) πŸ”—.
@@ -43,11 +59,12 @@ They are trained to classify speech as either "Real" 🟒 or "Fake" πŸ”΄.
43
  ⚠️ **Important:**
44
  To run inference with this model, you need to install a specific version of `fairseq` and make manual code modifications. For detailed instructions, please refer to the installation guide in our [GitHub repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake).
45
 
46
- For a quicker inference experience, we recommend trying our other SSL-AntiDeepfake models:
 
47
  - [MMS-300M-AntiDeepfake](https://huggingface.co/nii-yamagishilab/mms-300m-anti-deepfake)
48
  - [MMS-1B-AntiDeepfake](https://huggingface.co/nii-yamagishilab/mms-1b-anti-deepfake)
49
 
50
- πŸ“¦ Dependencies after manual code modifications:
51
  ```
52
  pip install huggingface-hub safetensors soundfile
53
  (pip install huggingface-hub==0.31.1 safetensors==0.5.3 soundfile==0.13.1)
@@ -62,6 +79,9 @@ import torchaudio
62
  import fairseq
63
  from huggingface_hub import PyTorchModelHubMixin
64
 
 
 
 
65
 
66
  # === Download Fairseq checkpoint if not present ===
67
  # The downloaded checkpoint is used for building front-end architecture,
@@ -138,13 +158,10 @@ model = DeepfakeDetector.from_pretrained("nii-yamagishilab/wav2vec-large-anti-de
138
  model.eval()
139
 
140
  # === Inference on a folder of audio files ===
141
- folder_path = "path/to/folder/contains/wavs/"
142
- supported_formats = (".mp3", ".wav", ".flac", ".m4a")
143
-
144
  results = []
145
  for root, _, files in os.walk(folder_path):
146
  for file in files:
147
- if file.lower().endswith(supported_formats):
148
  input_path = os.path.join(root, file)
149
  with torch.no_grad():
150
  wav = load_wav_and_preprocess(input_path)
@@ -161,6 +178,8 @@ for file_name, prob in results:
161
  print(f"{file_name}: real prob = {prob[1]:.3f}, fake prob = {prob[0]:.3f}")
162
  ```
163
  # πŸ“Š **Performance Metrics**
 
 
164
  | Test Database | ROC AUC | Accuracy | Precision | Recall | F1-score | FPR | FNR | EER (%) @ Threshold |
165
  |----------------------|---------|----------|-----------|--------|----------|-------|-------|----------------------|
166
  | ADD2023 | 0.950 | 0.912 | 0.939 | 0.942 | 0.940 | 0.175 | 0.058 | 13.25 @ 0.8520 |
@@ -171,8 +190,9 @@ for file_name, prob in results:
171
  | WildSVDD-TEST_A | 0.739 | 0.663 | 0.616 | 0.895 | 0.729 | 0.577 | 0.105 | 34.85 @ 0.9942 |
172
  | WildSVDD-TEST_B | 0.654 | 0.696 | 0.740 | 0.836 | 0.785 | 0.583 | 0.164 | 41.67 @ 0.9842 |
173
 
174
-
175
  # **Training Set**
 
 
176
  | πŸ“š Database | 🌍 Language | βœ… Genuine (hrs) | ❌ Fake (hrs) |
177
  |----------------------|------------------|----------------|-------------|
178
  | AISHELL3 | zh | 85.62 | 0 |
@@ -204,4 +224,17 @@ for file_name, prob in results:
204
  | VoxCeleb2 | Multilingual | 1179.62 | 0 |
205
  | VoxCeleb2-Vocoded | Multilingual | 0 | 4721.46 |
206
  | WaveFake | en, ja | 0 | 198.65 |
207
- | Train Set | Over 100 languages| 56370.00 | 18280.00 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
 
13
  # πŸ” **SSL-AntiDeepfake**
14
 
15
+ The **SSL-AntiDeepfake** project provides a series of powerful self-supervised learning (SSL) models crafted to **detect deepfake speech** with state-of-the-art accuracy.
 
 
 
 
16
 
17
 
18
+ # πŸ€– Available Models
19
+
20
+ All models are released on Hugging Face πŸ€— with two variants:
21
+ - **Default**: Trained with data augmentation
22
+ - **NDA** (No Data Augmentation): Trained without data augmentation
23
+
24
+ | Model | Variants |
25
+ |-----------------------------------------------|--------------------|
26
+ | XLS-R-2B-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/xls-r-2b-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/xls-r-2b-anti-deepfake-nda)
27
+ | XLS-R-1B-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/xls-r-1b-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/xls-r-1b-anti-deepfake-nda)
28
+ | MMS-1B-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/mms-1b-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/mms-1b-anti-deepfake-nda)
29
+ | MMS-300M-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/mms-300m-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/mms-300m-anti-deepfake-nda)
30
+ | Wav2Vec2-Large-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/wav2vec-large-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/wav2vec-large-anti-deepfake-nda)
31
+ | Wav2Vec2-Small-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/wav2vec-small-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/wav2vec-small-anti-deepfake-nda)
32
+ | Hubert-Extra-Large-AntiDeepfake | [Default](https://huggingface.co/nii-yamagishilab/hubert-xlarge-anti-deepfake), [NDA](https://huggingface.co/nii-yamagishilab/hubert-xlarge-anti-deepfake-nda)
33
+
34
+ # πŸ› οΈ Training Code & Repository
35
+ Explore training scripts, config files, and evaluation utilities in our GitHub repository:πŸ”— [SSL-AntiDeepfake GitHub Repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake)
36
+
37
+
38
+ # πŸš€ **Model Spotlight: Wav2Vec2-Large-AntiDeepfake**
39
 
40
  # 🌟 **Key Features**
41
  - **Architecture**: Wav2Vec 2.0 - [`facebook/wav2vec2-large-960h-lv60-self `](https://huggingface.co/facebook/wav2vec2-large-960h-lv60-self/tree/main) πŸ”—.
 
59
  ⚠️ **Important:**
60
  To run inference with this model, you need to install a specific version of `fairseq` and make manual code modifications. For detailed instructions, please refer to the installation guide in our [GitHub repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake).
61
 
62
+ βœ… **Easier Alternatives:**
63
+ We recommend these models for plug-and-play inference:
64
  - [MMS-300M-AntiDeepfake](https://huggingface.co/nii-yamagishilab/mms-300m-anti-deepfake)
65
  - [MMS-1B-AntiDeepfake](https://huggingface.co/nii-yamagishilab/mms-1b-anti-deepfake)
66
 
67
+ πŸ“¦ Dependencies after installing fairseq:
68
  ```
69
  pip install huggingface-hub safetensors soundfile
70
  (pip install huggingface-hub==0.31.1 safetensors==0.5.3 soundfile==0.13.1)
 
79
  import fairseq
80
  from huggingface_hub import PyTorchModelHubMixin
81
 
82
+ # === Detect all audio files in a specified folder ===
83
+ folder_path = "/path/to/folder/contains/wavs/"
84
+ audio_formats = (".mp3", ".wav", ".flac", ".m4a")
85
 
86
  # === Download Fairseq checkpoint if not present ===
87
  # The downloaded checkpoint is used for building front-end architecture,
 
158
  model.eval()
159
 
160
  # === Inference on a folder of audio files ===
 
 
 
161
  results = []
162
  for root, _, files in os.walk(folder_path):
163
  for file in files:
164
+ if file.lower().endswith(audio_formats):
165
  input_path = os.path.join(root, file)
166
  with torch.no_grad():
167
  wav = load_wav_and_preprocess(input_path)
 
178
  print(f"{file_name}: real prob = {prob[1]:.3f}, fake prob = {prob[0]:.3f}")
179
  ```
180
  # πŸ“Š **Performance Metrics**
181
+ Results shown below can be reproduced using scripts provided in our [GitHub repository](https://github.com/nii-yamagishilab/Ultra-SSL-AntiDeepfake).
182
+
183
  | Test Database | ROC AUC | Accuracy | Precision | Recall | F1-score | FPR | FNR | EER (%) @ Threshold |
184
  |----------------------|---------|----------|-----------|--------|----------|-------|-------|----------------------|
185
  | ADD2023 | 0.950 | 0.912 | 0.939 | 0.942 | 0.940 | 0.175 | 0.058 | 13.25 @ 0.8520 |
 
190
  | WildSVDD-TEST_A | 0.739 | 0.663 | 0.616 | 0.895 | 0.729 | 0.577 | 0.105 | 34.85 @ 0.9942 |
191
  | WildSVDD-TEST_B | 0.654 | 0.696 | 0.740 | 0.836 | 0.785 | 0.583 | 0.164 | 41.67 @ 0.9842 |
192
 
 
193
  # **Training Set**
194
+ Below is a breakdown of the training set used for fine-tuning.
195
+
196
  | πŸ“š Database | 🌍 Language | βœ… Genuine (hrs) | ❌ Fake (hrs) |
197
  |----------------------|------------------|----------------|-------------|
198
  | AISHELL3 | zh | 85.62 | 0 |
 
224
  | VoxCeleb2 | Multilingual | 1179.62 | 0 |
225
  | VoxCeleb2-Vocoded | Multilingual | 0 | 4721.46 |
226
  | WaveFake | en, ja | 0 | 198.65 |
227
+ | Train Set | Over 100 languages| 56370.00 | 18280.00 |
228
+
229
+ # **Attribution**
230
+ All SSL-AntiDeepfake models were developed by [Yamagishi Lab](https://yamagishilab.jp/) at the National Institute of Informatics (NII), Japan.
231
+
232
+ All model weights are the intellectual property of NII and are made available for research and educational purposes under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license.
233
+
234
+ # **Acknowledgments**
235
+ This project is based on results obtained from project JPNP22007, commissioned by the New Energy and Industrial Technology Development Organization (NEDO).
236
+
237
+ It is also partially supported by the following grants from the Japan Science and Technology Agency (JST):
238
+ - AIP Acceleration Research (Grant No. JPMJCR24U3)
239
+ - CREST (Grant No. JPMJCR20D3)
240
+ - PRESTO (Grant No. JPMJPR23P9)