nii-yamagishilab commited on
Commit
502c8bb
Β·
verified Β·
1 Parent(s): 5a949e6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -6
README.md CHANGED
@@ -10,10 +10,11 @@ base_model:
10
  - facebook/wav2vec2-large-960h-lv60-self
11
  ---
12
 
 
13
  # πŸ” **AntiDeepfake**
14
 
15
- The **AntiDeepfake** project provides a series of powerful self-supervised learning (SSL) models crafted to **detect deepfake speech** with state-of-the-art accuracy.
16
 
 
17
 
18
  # πŸ€– Available Models
19
 
@@ -195,11 +196,24 @@ Results shown below can be reproduced using scripts provided in our [GitHub repo
195
  | FakeOrReal | 1.000 | 0.992 | 0.994 | 0.989 | 0.991 | 0.005 | 0.011 | 0.63 @ 0.3727 |
196
  | FakeOrReal-norm | 0.999 | 0.986 | 0.975 | 0.997 | 0.986 | 0.025 | 0.003 | 0.97 @ 0.7975 |
197
  | In-the-Wild | 0.997 | 0.976 | 0.991 | 0.970 | 0.980 | 0.015 | 0.030 | 1.91 @ 0.3240 |
198
- | WildSVDD-TEST_A | 0.739 | 0.663 | 0.616 | 0.895 | 0.729 | 0.577 | 0.105 | 34.85 @ 0.9942 |
199
- | WildSVDD-TEST_B | 0.654 | 0.696 | 0.740 | 0.836 | 0.785 | 0.583 | 0.164 | 41.67 @ 0.9842 |
 
 
 
 
 
 
 
 
 
 
 
 
 
200
 
201
  # **Training Set**
202
- Below is a breakdown of the training set used for fine-tuning.
203
 
204
  | πŸ“š Database | 🌍 Language | βœ… Genuine (hrs) | ❌ Fake (hrs) |
205
  |----------------------|------------------|----------------|-------------|
@@ -244,5 +258,6 @@ This project is based on results obtained from project JPNP22007, commissioned b
244
 
245
  It is also partially supported by the following grants from the Japan Science and Technology Agency (JST):
246
  - AIP Acceleration Research (Grant No. JPMJCR24U3)
247
- - CREST (Grant No. JPMJCR20D3)
248
- - PRESTO (Grant No. JPMJPR23P9)
 
 
10
  - facebook/wav2vec2-large-960h-lv60-self
11
  ---
12
 
13
+
14
  # πŸ” **AntiDeepfake**
15
 
 
16
 
17
+ The **AntiDeepfake** project provides a series of powerful foundation models post-trained for **deepfake detection**. The AntiDeepfake model can be used for feature extraction for deepfake detection in a zero-shot manner, or it may be further fine-tuned and optimized for a specific database or deepfake-related task.
18
 
19
  # πŸ€– Available Models
20
 
 
196
  | FakeOrReal | 1.000 | 0.992 | 0.994 | 0.989 | 0.991 | 0.005 | 0.011 | 0.63 @ 0.3727 |
197
  | FakeOrReal-norm | 0.999 | 0.986 | 0.975 | 0.997 | 0.986 | 0.025 | 0.003 | 0.97 @ 0.7975 |
198
  | In-the-Wild | 0.997 | 0.976 | 0.991 | 0.970 | 0.980 | 0.015 | 0.030 | 1.91 @ 0.3240 |
199
+ | Deepfake-Eval-2024 | 0.741 | 0.714 | 0.720 | 0.920 | 0.808 | 0.675 | 0.081 | 33.38 @ 0.9995 |
200
+
201
+ You can also fine-tune this model on a specific database, the corresponding code is provided in our [GitHub repository](https://github.com/nii-yamagishilab/AntiDeepfake). Fine-tuning will follow a similar process to training a new model, except that model weights will be initialized as AntiDeepfake checkpoints.
202
+
203
+ Below are the evaluation results of this model fine-tuned on the Deepfake-Eval-2024 training set and tested on its corresponding test set (as shown in the previous table):
204
+
205
+ | Test Input Length | ROC AUC | Accuracy | Precision | Recall | F1-score | FPR | FNR | EER (%) @ Threshold |
206
+ |-------------------|---------|----------|-----------|--------|----------|--------|--------|----------------------|
207
+ | 4s | 0.8776 | 0.8147 | 0.8126 | 0.9315 | 0.8680 | 0.4060 | 0.0685 | 19.56 @ 0.9238 |
208
+ | 10s | 0.9362 | 0.8831 | 0.8914 | 0.9346 | 0.9125 | 0.2135 | 0.0654 | 12.10 @ 0.8521 |
209
+ | 13s | 0.9440 | 0.8977 | 0.9114 | 0.9337 | 0.9224 | 0.1697 | 0.0663 | 10.94 @ 0.8127 |
210
+ | 30s | 0.9495 | 0.8975 | 0.9310 | 0.9088 | 0.9198 | 0.1233 | 0.0912 | 10.52 @ 0.6061 |
211
+ | 50s | 0.9414 | 0.8792 | 0.9348 | 0.8707 | 0.9016 | 0.1060 | 0.1293 | 11.37 @ 0.4168 |
212
+
213
+
214
 
215
  # **Training Set**
216
+ Below is a breakdown of the training set used for post-training of speech SSL models.
217
 
218
  | πŸ“š Database | 🌍 Language | βœ… Genuine (hrs) | ❌ Fake (hrs) |
219
  |----------------------|------------------|----------------|-------------|
 
258
 
259
  It is also partially supported by the following grants from the Japan Science and Technology Agency (JST):
260
  - AIP Acceleration Research (Grant No. JPMJCR24U3)
261
+ - PRESTO (Grant No. JPMJPR23P9)
262
+
263
+ This study was carried out using the TSUBAME4.0 supercomputer at Institute of Science Tokyo.