Hugging Face's logo

0x3
/
smart-turn-v3

Voice Activity Detection
ONNX
speech-processing
semantic-vad
multilingual
Model card Files
xet
 Community
0x3 marcus-daily commited on
Commit
3d3c21c
·
0 Parent(s):

Duplicate from pipecat-ai/smart-turn-v3

Browse files

Co-authored-by: Marcus <marcus-daily@users.noreply.huggingface.co>

Files changed (12) hide show
  1. .gitattributes +35 -0
  2. README.md +44 -0
  3. benchmarks/smart-turn-v3.0.md +65 -0
  4. benchmarks/smart-turn-v3.1-cpu.md +65 -0
  5. benchmarks/smart-turn-v3.1-gpu.md +65 -0
  6. benchmarks/smart-turn-v3.2-cpu.md +65 -0
  7. benchmarks/smart-turn-v3.2-gpu.md +65 -0
  8. smart-turn-v3.0.onnx +3 -0
  9. smart-turn-v3.1-cpu.onnx +3 -0
  10. smart-turn-v3.1-gpu.onnx +3 -0
  11. smart-turn-v3.2-cpu.onnx +3 -0
  12. smart-turn-v3.2-gpu.onnx +3 -0
.gitattributes ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: voice-activity-detection
3
+ license: bsd-2-clause
4
+ tags:
5
+ - speech-processing
6
+ - semantic-vad
7
+ - multilingual
8
+ datasets:
9
+ - pipecat-ai/smart-turn-data-v3.1-train
10
+ - pipecat-ai/smart-turn-data-v3.1-test
11
+ ---
12
+
13
+ # Smart Turn v3.x
14
+
15
+ **Smart Turn** is an open‑source semantic Voice Activity Detection (VAD) model that tells you whether a speaker has finished their turn by analysing the raw waveform, not the transcript.
16
+
17
+ ## Links
18
+
19
+ * [Blog post: Smart Turn v3](https://www.daily.co/blog/announcing-smart-turn-v3-with-cpu-inference-in-just-12ms/)
20
+ * [GitHub repo](https://github.com/pipecat-ai/smart-turn) with training and inference code, and more information
21
+ * [Datasets](https://huggingface.co/pipecat-ai/datasets)
22
+
23
+
24
+ ## Model architecture
25
+
26
+ * Backbone: Whisper Tiny encoder
27
+ * Head: shallow linear classifier
28
+ * Params: 8M
29
+ * Checkpoint: 8 MB ONNX (int8 quantized), 32MB ONNX (unquantized)
30
+
31
+
32
+ ## How to use
33
+
34
+ Please see the blog post and GitHub repo for more information on using the model, either standalone or with Pipecat.
35
+
36
+
37
+ ## Thanks
38
+
39
+ Thank you to the following organisations for contributing audio datasets:
40
+
41
+ - [Liva AI](https://www.theliva.ai/)
42
+ - [Midcentury](https://www.midcentury.xyz/)
43
+ - [MundoAI](https://mundoai.world/)
44
+
benchmarks/smart-turn-v3.0.md ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Endpointing Model Benchmark Report
2
+
3
+ **Model:** `/data/smart-turn-v3.0.onnx`
4
+
5
+ **Generated:** 2026-01-07 17:31:01 UTC
6
+
7
+ ## Accuracy Results
8
+
9
+ **Total Samples:** 31,527
10
+
11
+ **Unique Languages:** 🇸🇦 Arabic, 🇧🇩 Bengali, 🇩🇰 Danish, 🇩🇪 German, 🇬🇧 🇺🇸 English, 🇫🇮 Finnish, 🇫🇷 French, 🇮🇳 Hindi, 🇮🇩 Indonesian, 🇮🇹 Italian, 🇯🇵 Japanese, 🇰🇷 Korean, 🇮🇳 Marathi, 🇳🇱 Dutch, 🇳🇴 Norwegian, 🇵🇱 Polish, 🇵🇹 Portuguese, 🇷🇺 Russian, 🇪🇸 Spanish, 🇹🇷 Turkish, 🇺🇦 Ukrainian, 🇻🇳 Vietnamese, 🇨🇳 Chinese
12
+
13
+ **Unique Datasets:** chirp3_1, chirp3_2, chirp3_3_short, human_5, human_convcollector_1, liva_1, midcentury_1, mundo_1, orpheus_endfiller_1, orpheus_grammar_1, orpheus_midfiller_1, rime_2
14
+
15
+ ### Overall Performance
16
+
17
+ | Metric | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
18
+ | :------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
19
+ | Overall | 31,527 | 88.97 | 0.858 | 0.933 | 0.894 | 7.70 | 3.33 |
20
+
21
+ ### Performance by Language
22
+
23
+ | Language | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
24
+ | :------------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
25
+ | 🇰🇷 Korean | 889 | 95.39 | 0.947 | 0.962 | 0.954 | 2.70 | 1.91 |
26
+ | 🇹🇷 Turkish | 966 | 95.34 | 0.927 | 0.983 | 0.954 | 3.83 | 0.83 |
27
+ | 🇩🇪 German | 1,322 | 95.23 | 0.928 | 0.980 | 0.954 | 3.78 | 0.98 |
28
+ | 🇵🇹 Portuguese | 1,398 | 94.49 | 0.927 | 0.962 | 0.944 | 3.65 | 1.86 |
29
+ | 🇯🇵 Japanese | 834 | 94.36 | 0.925 | 0.967 | 0.945 | 3.96 | 1.68 |
30
+ | 🇳🇱 Dutch | 1,398 | 94.28 | 0.925 | 0.968 | 0.946 | 4.08 | 1.65 |
31
+ | 🇫🇷 French | 1,252 | 94.25 | 0.945 | 0.942 | 0.944 | 2.80 | 2.96 |
32
+ | 🇮🇹 Italian | 782 | 93.22 | 0.898 | 0.974 | 0.935 | 5.50 | 1.28 |
33
+ | 🇵🇱 Polish | 974 | 93.02 | 0.904 | 0.955 | 0.929 | 4.83 | 2.16 |
34
+ | 🇷🇺 Russian | 1,468 | 92.03 | 0.891 | 0.966 | 0.927 | 6.20 | 1.77 |
35
+ | 🇺🇦 Ukrainian | 929 | 91.82 | 0.881 | 0.954 | 0.916 | 6.03 | 2.15 |
36
+ | 🇮🇩 Indonesian | 971 | 90.73 | 0.854 | 0.979 | 0.912 | 8.24 | 1.03 |
37
+ | 🇮🇳 Hindi | 1,284 | 89.95 | 0.856 | 0.970 | 0.910 | 8.49 | 1.56 |
38
+ | 🇩🇰 Danish | 779 | 89.73 | 0.841 | 0.982 | 0.906 | 9.37 | 0.90 |
39
+ | 🇳🇴 Norwegian | 1,014 | 89.64 | 0.869 | 0.938 | 0.903 | 7.20 | 3.16 |
40
+ | 🇨🇳 Chinese | 929 | 89.34 | 0.861 | 0.943 | 0.900 | 7.75 | 2.91 |
41
+ | 🇫🇮 Finnish | 1,010 | 87.52 | 0.813 | 0.974 | 0.886 | 11.19 | 1.29 |
42
+ | 🇸🇦 Arabic | 947 | 86.17 | 0.810 | 0.950 | 0.875 | 11.30 | 2.53 |
43
+ | 🇬🇧 🇺🇸 English | 7,820 | 85.49 | 0.836 | 0.876 | 0.855 | 8.41 | 6.10 |
44
+ | 🇪🇸 Spanish | 1,783 | 84.46 | 0.848 | 0.831 | 0.839 | 7.29 | 8.24 |
45
+ | 🇮🇳 Marathi | 774 | 81.01 | 0.751 | 0.936 | 0.834 | 15.76 | 3.23 |
46
+ | 🇻🇳 Vietnamese | 1,004 | 79.58 | 0.721 | 0.960 | 0.824 | 18.43 | 1.99 |
47
+ | 🇧🇩 Bengali | 1,000 | 78.70 | 0.717 | 0.935 | 0.811 | 18.10 | 3.20 |
48
+
49
+ ### Performance by Dataset
50
+
51
+ | Dataset | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
52
+ | :-------------------- | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
53
+ | rime_2 | 394 | 95.94 | 0.981 | 0.922 | 0.951 | 0.76 | 3.30 |
54
+ | orpheus_endfiller_1 | 181 | 95.03 | 1.000 | 0.902 | 0.949 | 0.00 | 4.97 |
55
+ | human_5 | 402 | 94.03 | 0.948 | 0.916 | 0.931 | 2.24 | 3.73 |
56
+ | chirp3_1 | 16,254 | 92.84 | 0.903 | 0.961 | 0.931 | 5.19 | 1.96 |
57
+ | human_convcollector_1 | 90 | 91.11 | 0.895 | 0.895 | 0.895 | 4.44 | 4.44 |
58
+ | orpheus_midfiller_1 | 140 | 90.00 | 0.877 | 0.905 | 0.891 | 5.71 | 4.29 |
59
+ | orpheus_grammar_1 | 163 | 87.12 | 0.848 | 0.918 | 0.881 | 8.59 | 4.29 |
60
+ | chirp3_2 | 8,428 | 86.08 | 0.802 | 0.956 | 0.872 | 11.75 | 2.17 |
61
+ | liva_1 | 3,831 | 83.74 | 0.839 | 0.838 | 0.838 | 8.09 | 8.17 |
62
+ | midcentury_1 | 1,044 | 77.39 | 0.704 | 0.917 | 0.797 | 18.58 | 4.02 |
63
+ | chirp3_3_short | 104 | 74.04 | 0.818 | 0.562 | 0.667 | 5.77 | 20.19 |
64
+ | mundo_1 | 496 | 67.34 | 0.741 | 0.524 | 0.614 | 9.07 | 23.59 |
65
+
benchmarks/smart-turn-v3.1-cpu.md ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Endpointing Model Benchmark Report
2
+
3
+ **Model:** `/data/smart-turn-v3.1-cpu.onnx`
4
+
5
+ **Generated:** 2026-01-07 17:38:46 UTC
6
+
7
+ ## Accuracy Results
8
+
9
+ **Total Samples:** 31,527
10
+
11
+ **Unique Languages:** 🇸🇦 Arabic, 🇧🇩 Bengali, 🇩🇰 Danish, 🇩🇪 German, 🇬🇧 🇺🇸 English, 🇫🇮 Finnish, 🇫🇷 French, 🇮🇳 Hindi, 🇮🇩 Indonesian, 🇮🇹 Italian, 🇯🇵 Japanese, 🇰🇷 Korean, 🇮🇳 Marathi, 🇳🇱 Dutch, 🇳🇴 Norwegian, 🇵🇱 Polish, 🇵🇹 Portuguese, 🇷🇺 Russian, 🇪🇸 Spanish, 🇹🇷 Turkish, 🇺🇦 Ukrainian, 🇻🇳 Vietnamese, 🇨🇳 Chinese
12
+
13
+ **Unique Datasets:** chirp3_1, chirp3_2, chirp3_3_short, human_5, human_convcollector_1, liva_1, midcentury_1, mundo_1, orpheus_endfiller_1, orpheus_grammar_1, orpheus_midfiller_1, rime_2
14
+
15
+ ### Overall Performance
16
+
17
+ | Metric | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
18
+ | :------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
19
+ | Overall | 31,527 | 90.13 | 0.883 | 0.924 | 0.903 | 6.08 | 3.79 |
20
+
21
+ ### Performance by Language
22
+
23
+ | Language | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
24
+ | :------------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
25
+ | 🇯🇵 Japanese | 834 | 95.32 | 0.936 | 0.974 | 0.954 | 3.36 | 1.32 |
26
+ | 🇫🇷 French | 1,252 | 94.89 | 0.949 | 0.951 | 0.950 | 2.64 | 2.48 |
27
+ | 🇳🇱 Dutch | 1,398 | 94.71 | 0.943 | 0.956 | 0.949 | 3.00 | 2.29 |
28
+ | 🇩🇪 German | 1,322 | 94.02 | 0.928 | 0.954 | 0.941 | 3.71 | 2.27 |
29
+ | 🇵🇱 Polish | 974 | 93.43 | 0.910 | 0.957 | 0.933 | 4.52 | 2.05 |
30
+ | 🇰🇷 Korean | 889 | 93.03 | 0.944 | 0.914 | 0.929 | 2.70 | 4.27 |
31
+ | 🇵🇹 Portuguese | 1,398 | 92.85 | 0.946 | 0.904 | 0.925 | 2.50 | 4.65 |
32
+ | 🇹🇷 Turkish | 966 | 92.75 | 0.900 | 0.960 | 0.929 | 5.28 | 1.97 |
33
+ | 🇮🇩 Indonesian | 971 | 92.17 | 0.912 | 0.931 | 0.921 | 4.43 | 3.40 |
34
+ | 🇮🇹 Italian | 782 | 91.94 | 0.881 | 0.969 | 0.923 | 6.52 | 1.53 |
35
+ | 🇷🇺 Russian | 1,468 | 91.42 | 0.901 | 0.940 | 0.920 | 5.45 | 3.13 |
36
+ | 🇬🇧 🇺🇸 English | 7,820 | 90.66 | 0.885 | 0.930 | 0.907 | 5.92 | 3.41 |
37
+ | 🇩🇰 Danish | 779 | 90.63 | 0.876 | 0.949 | 0.911 | 6.80 | 2.57 |
38
+ | 🇮🇳 Hindi | 1,284 | 90.58 | 0.897 | 0.925 | 0.911 | 5.53 | 3.89 |
39
+ | 🇺🇦 Ukrainian | 929 | 89.99 | 0.869 | 0.927 | 0.897 | 6.57 | 3.44 |
40
+ | 🇳🇴 Norwegian | 1,014 | 88.95 | 0.855 | 0.944 | 0.897 | 8.19 | 2.86 |
41
+ | 🇫🇮 Finnish | 1,010 | 88.32 | 0.831 | 0.960 | 0.891 | 9.70 | 1.98 |
42
+ | 🇪🇸 Spanish | 1,783 | 87.49 | 0.854 | 0.898 | 0.875 | 7.52 | 4.99 |
43
+ | 🇸🇦 Arabic | 947 | 85.96 | 0.831 | 0.909 | 0.868 | 9.40 | 4.65 |
44
+ | 🇨🇳 Chinese | 929 | 85.25 | 0.833 | 0.888 | 0.860 | 9.04 | 5.71 |
45
+ | 🇮🇳 Marathi | 774 | 82.30 | 0.792 | 0.883 | 0.835 | 11.76 | 5.94 |
46
+ | 🇧🇩 Bengali | 1,000 | 80.30 | 0.774 | 0.845 | 0.808 | 12.10 | 7.60 |
47
+ | 🇻🇳 Vietnamese | 1,004 | 77.99 | 0.805 | 0.735 | 0.769 | 8.86 | 13.15 |
48
+
49
+ ### Performance by Dataset
50
+
51
+ | Dataset | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
52
+ | :-------------------- | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
53
+ | orpheus_endfiller_1 | 181 | 95.58 | 1.000 | 0.913 | 0.955 | 0.00 | 4.42 |
54
+ | liva_1 | 3,831 | 93.45 | 0.929 | 0.942 | 0.935 | 3.63 | 2.92 |
55
+ | human_5 | 402 | 93.03 | 0.931 | 0.910 | 0.920 | 2.99 | 3.98 |
56
+ | rime_2 | 394 | 92.89 | 0.932 | 0.898 | 0.915 | 2.79 | 4.31 |
57
+ | chirp3_1 | 16,254 | 92.09 | 0.907 | 0.938 | 0.923 | 4.82 | 3.09 |
58
+ | orpheus_grammar_1 | 163 | 89.57 | 0.895 | 0.906 | 0.901 | 5.52 | 4.91 |
59
+ | orpheus_midfiller_1 | 140 | 89.29 | 0.875 | 0.889 | 0.882 | 5.71 | 5.00 |
60
+ | chirp3_2 | 8,428 | 86.24 | 0.838 | 0.897 | 0.866 | 8.64 | 5.13 |
61
+ | human_convcollector_1 | 90 | 85.56 | 0.791 | 0.895 | 0.840 | 10.00 | 4.44 |
62
+ | mundo_1 | 496 | 82.06 | 0.815 | 0.825 | 0.820 | 9.27 | 8.67 |
63
+ | midcentury_1 | 1,044 | 81.32 | 0.742 | 0.940 | 0.829 | 15.80 | 2.87 |
64
+ | chirp3_3_short | 104 | 78.85 | 0.842 | 0.667 | 0.744 | 5.77 | 15.38 |
65
+
benchmarks/smart-turn-v3.1-gpu.md ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Endpointing Model Benchmark Report
2
+
3
+ **Model:** `/data/smart-turn-v3.1-gpu.onnx`
4
+
5
+ **Generated:** 2026-01-07 17:45:59 UTC
6
+
7
+ ## Accuracy Results
8
+
9
+ **Total Samples:** 31,527
10
+
11
+ **Unique Languages:** 🇸🇦 Arabic, 🇧🇩 Bengali, 🇩🇰 Danish, 🇩🇪 German, 🇬🇧 🇺🇸 English, 🇫🇮 Finnish, 🇫🇷 French, 🇮🇳 Hindi, 🇮🇩 Indonesian, 🇮🇹 Italian, 🇯🇵 Japanese, 🇰🇷 Korean, 🇮🇳 Marathi, 🇳🇱 Dutch, 🇳🇴 Norwegian, 🇵🇱 Polish, 🇵🇹 Portuguese, 🇷🇺 Russian, 🇪🇸 Spanish, 🇹🇷 Turkish, 🇺🇦 Ukrainian, 🇻🇳 Vietnamese, 🇨🇳 Chinese
12
+
13
+ **Unique Datasets:** chirp3_1, chirp3_2, chirp3_3_short, human_5, human_convcollector_1, liva_1, midcentury_1, mundo_1, orpheus_endfiller_1, orpheus_grammar_1, orpheus_midfiller_1, rime_2
14
+
15
+ ### Overall Performance
16
+
17
+ | Metric | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
18
+ | :------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
19
+ | Overall | 31,527 | 91.64 | 0.894 | 0.944 | 0.918 | 5.55 | 2.81 |
20
+
21
+ ### Performance by Language
22
+
23
+ | Language | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
24
+ | :------------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
25
+ | 🇯🇵 Japanese | 834 | 95.68 | 0.944 | 0.971 | 0.958 | 2.88 | 1.44 |
26
+ | 🇳🇱 Dutch | 1,398 | 95.42 | 0.950 | 0.963 | 0.956 | 2.65 | 1.93 |
27
+ | 🇹🇷 Turkish | 966 | 95.34 | 0.935 | 0.973 | 0.954 | 3.31 | 1.35 |
28
+ | 🇫🇷 French | 1,252 | 95.29 | 0.950 | 0.958 | 0.954 | 2.56 | 2.16 |
29
+ | 🇰🇷 Korean | 889 | 95.16 | 0.965 | 0.937 | 0.951 | 1.69 | 3.15 |
30
+ | 🇩🇪 German | 1,322 | 95.16 | 0.936 | 0.970 | 0.952 | 3.33 | 1.51 |
31
+ | 🇵🇹 Portuguese | 1,398 | 94.85 | 0.942 | 0.953 | 0.947 | 2.86 | 2.29 |
32
+ | 🇮🇹 Italian | 782 | 94.50 | 0.922 | 0.972 | 0.946 | 4.09 | 1.41 |
33
+ | 🇵🇱 Polish | 974 | 94.35 | 0.921 | 0.963 | 0.942 | 3.90 | 1.75 |
34
+ | 🇮🇩 Indonesian | 971 | 93.10 | 0.905 | 0.960 | 0.932 | 4.94 | 1.96 |
35
+ | 🇷🇺 Russian | 1,468 | 92.64 | 0.911 | 0.953 | 0.932 | 4.90 | 2.45 |
36
+ | 🇮🇳 Hindi | 1,284 | 92.52 | 0.919 | 0.939 | 0.929 | 4.28 | 3.19 |
37
+ | 🇺🇦 Ukrainian | 929 | 92.03 | 0.900 | 0.933 | 0.917 | 4.84 | 3.12 |
38
+ | 🇬🇧 🇺🇸 English | 7,820 | 91.94 | 0.889 | 0.954 | 0.921 | 5.82 | 2.24 |
39
+ | 🇩🇰 Danish | 779 | 91.14 | 0.880 | 0.954 | 0.916 | 6.55 | 2.31 |
40
+ | 🇫🇮 Finnish | 1,010 | 90.50 | 0.859 | 0.968 | 0.910 | 7.92 | 1.58 |
41
+ | 🇳🇴 Norwegian | 1,014 | 89.84 | 0.865 | 0.950 | 0.905 | 7.59 | 2.56 |
42
+ | 🇪🇸 Spanish | 1,783 | 89.62 | 0.871 | 0.924 | 0.897 | 6.67 | 3.70 |
43
+ | 🇨🇳 Chinese | 929 | 88.37 | 0.850 | 0.937 | 0.891 | 8.40 | 3.23 |
44
+ | 🇸🇦 Arabic | 947 | 87.01 | 0.838 | 0.923 | 0.878 | 9.08 | 3.91 |
45
+ | 🇮🇳 Marathi | 774 | 84.88 | 0.833 | 0.878 | 0.855 | 8.91 | 6.20 |
46
+ | 🇧🇩 Bengali | 1,000 | 81.20 | 0.801 | 0.820 | 0.810 | 10.00 | 8.80 |
47
+ | 🇻🇳 Vietnamese | 1,004 | 81.08 | 0.780 | 0.862 | 0.819 | 12.05 | 6.87 |
48
+
49
+ ### Performance by Dataset
50
+
51
+ | Dataset | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
52
+ | :-------------------- | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
53
+ | orpheus_endfiller_1 | 181 | 97.24 | 1.000 | 0.946 | 0.972 | 0.00 | 2.76 |
54
+ | rime_2 | 394 | 97.21 | 0.959 | 0.976 | 0.967 | 1.78 | 1.02 |
55
+ | human_5 | 402 | 95.02 | 0.939 | 0.949 | 0.944 | 2.74 | 2.24 |
56
+ | liva_1 | 3,831 | 94.23 | 0.929 | 0.959 | 0.944 | 3.68 | 2.09 |
57
+ | chirp3_1 | 16,254 | 93.53 | 0.919 | 0.955 | 0.937 | 4.22 | 2.25 |
58
+ | orpheus_grammar_1 | 163 | 89.57 | 0.878 | 0.929 | 0.903 | 6.75 | 3.68 |
59
+ | orpheus_midfiller_1 | 140 | 89.29 | 0.853 | 0.921 | 0.885 | 7.14 | 3.57 |
60
+ | chirp3_2 | 8,428 | 87.81 | 0.850 | 0.916 | 0.882 | 8.03 | 4.15 |
61
+ | chirp3_3_short | 104 | 85.58 | 0.867 | 0.812 | 0.839 | 5.77 | 8.65 |
62
+ | mundo_1 | 496 | 84.68 | 0.840 | 0.854 | 0.847 | 8.06 | 7.26 |
63
+ | human_convcollector_1 | 90 | 84.44 | 0.761 | 0.921 | 0.833 | 12.22 | 3.33 |
64
+ | midcentury_1 | 1,044 | 84.39 | 0.766 | 0.974 | 0.858 | 14.37 | 1.25 |
65
+
benchmarks/smart-turn-v3.2-cpu.md ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Endpointing Model Benchmark Report
2
+
3
+ **Model:** `/data/smart-turn-v3.2-cpu.onnx`
4
+
5
+ **Generated:** 2026-01-07 17:53:34 UTC
6
+
7
+ ## Accuracy Results
8
+
9
+ **Total Samples:** 31,527
10
+
11
+ **Unique Languages:** 🇸🇦 Arabic, 🇧🇩 Bengali, 🇩🇰 Danish, 🇩🇪 German, 🇬🇧 🇺🇸 English, 🇫🇮 Finnish, 🇫🇷 French, 🇮🇳 Hindi, 🇮🇩 Indonesian, 🇮🇹 Italian, 🇯🇵 Japanese, 🇰🇷 Korean, 🇮🇳 Marathi, 🇳🇱 Dutch, 🇳🇴 Norwegian, 🇵🇱 Polish, 🇵🇹 Portuguese, 🇷🇺 Russian, 🇪🇸 Spanish, 🇹🇷 Turkish, 🇺🇦 Ukrainian, 🇻🇳 Vietnamese, 🇨🇳 Chinese
12
+
13
+ **Unique Datasets:** chirp3_1, chirp3_2, chirp3_3_short, human_5, human_convcollector_1, liva_1, midcentury_1, mundo_1, orpheus_endfiller_1, orpheus_grammar_1, orpheus_midfiller_1, rime_2
14
+
15
+ ### Overall Performance
16
+
17
+ | Metric | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
18
+ | :------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
19
+ | Overall | 31,527 | 92.63 | 0.909 | 0.947 | 0.927 | 4.73 | 2.64 |
20
+
21
+ ### Performance by Language
22
+
23
+ | Language | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
24
+ | :------------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
25
+ | 🇰🇷 Korean | 889 | 96.96 | 0.956 | 0.984 | 0.970 | 2.25 | 0.79 |
26
+ | 🇹🇷 Turkish | 966 | 96.79 | 0.955 | 0.981 | 0.968 | 2.28 | 0.93 |
27
+ | 🇩🇪 German | 1,322 | 96.37 | 0.947 | 0.982 | 0.964 | 2.72 | 0.91 |
28
+ | 🇯🇵 Japanese | 834 | 96.16 | 0.962 | 0.962 | 0.962 | 1.92 | 1.92 |
29
+ | 🇳🇱 Dutch | 1,398 | 95.92 | 0.954 | 0.968 | 0.961 | 2.43 | 1.65 |
30
+ | 🇵🇱 Polish | 974 | 95.38 | 0.948 | 0.955 | 0.952 | 2.46 | 2.16 |
31
+ | 🇮🇩 Indonesian | 971 | 95.16 | 0.939 | 0.964 | 0.951 | 3.09 | 1.75 |
32
+ | 🇮🇹 Italian | 782 | 94.50 | 0.930 | 0.961 | 0.946 | 3.58 | 1.92 |
33
+ | 🇵🇹 Portuguese | 1,398 | 94.49 | 0.934 | 0.954 | 0.944 | 3.29 | 2.22 |
34
+ | 🇬🇧 🇺🇸 English | 7,820 | 94.26 | 0.926 | 0.959 | 0.942 | 3.75 | 1.99 |
35
+ | 🇺🇦 Ukrainian | 929 | 94.19 | 0.924 | 0.954 | 0.939 | 3.66 | 2.15 |
36
+ | 🇫🇮 Finnish | 1,010 | 94.16 | 0.930 | 0.954 | 0.942 | 3.56 | 2.28 |
37
+ | 🇫🇷 French | 1,252 | 94.09 | 0.924 | 0.964 | 0.943 | 4.07 | 1.84 |
38
+ | 🇷🇺 Russian | 1,468 | 93.53 | 0.920 | 0.960 | 0.940 | 4.36 | 2.11 |
39
+ | 🇩🇰 Danish | 779 | 92.81 | 0.906 | 0.957 | 0.931 | 5.01 | 2.18 |
40
+ | 🇳🇴 Norwegian | 1,014 | 91.81 | 0.896 | 0.950 | 0.922 | 5.62 | 2.56 |
41
+ | 🇮🇳 Hindi | 1,284 | 90.11 | 0.856 | 0.975 | 0.911 | 8.57 | 1.32 |
42
+ | 🇸🇦 Arabic | 947 | 89.76 | 0.872 | 0.936 | 0.903 | 6.97 | 3.27 |
43
+ | 🇪🇸 Spanish | 1,783 | 89.57 | 0.867 | 0.929 | 0.897 | 6.95 | 3.48 |
44
+ | 🇨🇳 Chinese | 929 | 85.79 | 0.894 | 0.818 | 0.854 | 4.95 | 9.26 |
45
+ | 🇧🇩 Bengali | 1,000 | 83.80 | 0.800 | 0.892 | 0.844 | 10.90 | 5.30 |
46
+ | 🇮🇳 Marathi | 774 | 82.43 | 0.762 | 0.952 | 0.846 | 15.12 | 2.45 |
47
+ | 🇻🇳 Vietnamese | 1,004 | 79.38 | 0.811 | 0.764 | 0.786 | 8.86 | 11.75 |
48
+
49
+ ### Performance by Dataset
50
+
51
+ | Dataset | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
52
+ | :-------------------- | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
53
+ | midcentury_1 | 1,044 | 98.47 | 0.978 | 0.990 | 0.984 | 1.05 | 0.48 |
54
+ | rime_2 | 394 | 96.95 | 0.970 | 0.958 | 0.964 | 1.27 | 1.78 |
55
+ | orpheus_endfiller_1 | 181 | 96.69 | 1.000 | 0.935 | 0.966 | 0.00 | 3.31 |
56
+ | human_5 | 402 | 96.02 | 0.945 | 0.966 | 0.956 | 2.49 | 1.49 |
57
+ | liva_1 | 3,831 | 93.97 | 0.924 | 0.960 | 0.941 | 3.99 | 2.04 |
58
+ | chirp3_1 | 16,254 | 93.78 | 0.923 | 0.955 | 0.939 | 3.97 | 2.25 |
59
+ | orpheus_grammar_1 | 163 | 91.41 | 0.890 | 0.953 | 0.920 | 6.13 | 2.45 |
60
+ | chirp3_3_short | 104 | 91.35 | 0.933 | 0.875 | 0.903 | 2.88 | 5.77 |
61
+ | chirp3_2 | 8,428 | 89.31 | 0.870 | 0.923 | 0.896 | 6.85 | 3.84 |
62
+ | orpheus_midfiller_1 | 140 | 87.14 | 0.846 | 0.873 | 0.859 | 7.14 | 5.71 |
63
+ | human_convcollector_1 | 90 | 86.67 | 0.810 | 0.895 | 0.850 | 8.89 | 4.44 |
64
+ | mundo_1 | 496 | 84.27 | 0.796 | 0.919 | 0.853 | 11.69 | 4.03 |
65
+
benchmarks/smart-turn-v3.2-gpu.md ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Endpointing Model Benchmark Report
2
+
3
+ **Model:** `/data/smart-turn-v3.2-gpu.onnx`
4
+
5
+ **Generated:** 2026-01-07 17:59:39 UTC
6
+
7
+ ## Accuracy Results
8
+
9
+ **Total Samples:** 31,527
10
+
11
+ **Unique Languages:** 🇸🇦 Arabic, 🇧🇩 Bengali, 🇩🇰 Danish, 🇩🇪 German, 🇬🇧 🇺🇸 English, 🇫🇮 Finnish, 🇫🇷 French, 🇮🇳 Hindi, 🇮🇩 Indonesian, 🇮🇹 Italian, 🇯🇵 Japanese, 🇰🇷 Korean, 🇮🇳 Marathi, 🇳🇱 Dutch, 🇳🇴 Norwegian, 🇵🇱 Polish, 🇵🇹 Portuguese, 🇷🇺 Russian, 🇪🇸 Spanish, 🇹🇷 Turkish, 🇺🇦 Ukrainian, 🇻🇳 Vietnamese, 🇨🇳 Chinese
12
+
13
+ **Unique Datasets:** chirp3_1, chirp3_2, chirp3_3_short, human_5, human_convcollector_1, liva_1, midcentury_1, mundo_1, orpheus_endfiller_1, orpheus_grammar_1, orpheus_midfiller_1, rime_2
14
+
15
+ ### Overall Performance
16
+
17
+ | Metric | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
18
+ | :------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
19
+ | Overall | 31,527 | 93.71 | 0.931 | 0.944 | 0.937 | 3.51 | 2.78 |
20
+
21
+ ### Performance by Language
22
+
23
+ | Language | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
24
+ | :------------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
25
+ | 🇰🇷 Korean | 889 | 97.64 | 0.977 | 0.975 | 0.976 | 1.12 | 1.24 |
26
+ | 🇯🇵 Japanese | 834 | 97.12 | 0.974 | 0.969 | 0.971 | 1.32 | 1.56 |
27
+ | 🇹🇷 Turkish | 966 | 97.00 | 0.967 | 0.973 | 0.970 | 1.66 | 1.35 |
28
+ | 🇳🇱 Dutch | 1,398 | 96.92 | 0.966 | 0.975 | 0.970 | 1.79 | 1.29 |
29
+ | 🇩🇪 German | 1,322 | 96.60 | 0.957 | 0.976 | 0.966 | 2.19 | 1.21 |
30
+ | 🇵🇹 Portuguese | 1,398 | 95.49 | 0.948 | 0.960 | 0.954 | 2.58 | 1.93 |
31
+ | 🇮🇩 Indonesian | 971 | 95.47 | 0.939 | 0.971 | 0.955 | 3.09 | 1.44 |
32
+ | 🇫🇮 Finnish | 1,010 | 95.25 | 0.950 | 0.954 | 0.952 | 2.48 | 2.28 |
33
+ | 🇵🇱 Polish | 974 | 95.17 | 0.946 | 0.952 | 0.949 | 2.57 | 2.26 |
34
+ | 🇺🇦 Ukrainian | 929 | 95.05 | 0.943 | 0.952 | 0.947 | 2.69 | 2.26 |
35
+ | 🇮🇹 Italian | 782 | 95.01 | 0.949 | 0.951 | 0.950 | 2.56 | 2.43 |
36
+ | 🇫🇷 French | 1,252 | 94.73 | 0.941 | 0.956 | 0.949 | 3.04 | 2.24 |
37
+ | 🇬🇧 🇺🇸 English | 7,820 | 94.71 | 0.940 | 0.953 | 0.946 | 2.98 | 2.31 |
38
+ | 🇷🇺 Russian | 1,468 | 94.41 | 0.937 | 0.958 | 0.947 | 3.41 | 2.18 |
39
+ | 🇩🇰 Danish | 779 | 93.58 | 0.930 | 0.944 | 0.937 | 3.59 | 2.82 |
40
+ | 🇳🇴 Norwegian | 1,014 | 93.00 | 0.929 | 0.934 | 0.932 | 3.65 | 3.35 |
41
+ | 🇮🇳 Hindi | 1,284 | 92.76 | 0.930 | 0.931 | 0.931 | 3.66 | 3.58 |
42
+ | 🇪🇸 Spanish | 1,783 | 91.53 | 0.908 | 0.920 | 0.914 | 4.54 | 3.93 |
43
+ | 🇨🇳 Chinese | 929 | 90.53 | 0.899 | 0.918 | 0.908 | 5.27 | 4.20 |
44
+ | 🇸🇦 Arabic | 947 | 89.12 | 0.869 | 0.925 | 0.896 | 7.07 | 3.80 |
45
+ | 🇮🇳 Marathi | 774 | 88.11 | 0.870 | 0.901 | 0.885 | 6.85 | 5.04 |
46
+ | 🇧🇩 Bengali | 1,000 | 85.10 | 0.847 | 0.849 | 0.848 | 7.50 | 7.40 |
47
+ | 🇻🇳 Vietnamese | 1,004 | 82.47 | 0.814 | 0.840 | 0.826 | 9.56 | 7.97 |
48
+
49
+ ### Performance by Dataset
50
+
51
+ | Dataset | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
52
+ | :-------------------- | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
53
+ | midcentury_1 | 1,044 | 98.85 | 0.992 | 0.984 | 0.988 | 0.38 | 0.77 |
54
+ | rime_2 | 394 | 98.22 | 0.982 | 0.976 | 0.979 | 0.76 | 1.02 |
55
+ | human_5 | 402 | 97.01 | 0.977 | 0.955 | 0.966 | 1.00 | 1.99 |
56
+ | orpheus_endfiller_1 | 181 | 95.58 | 0.988 | 0.924 | 0.955 | 0.55 | 3.87 |
57
+ | chirp3_1 | 16,254 | 94.80 | 0.943 | 0.954 | 0.948 | 2.89 | 2.31 |
58
+ | liva_1 | 3,831 | 94.49 | 0.934 | 0.958 | 0.946 | 3.39 | 2.11 |
59
+ | orpheus_grammar_1 | 163 | 92.02 | 0.919 | 0.929 | 0.924 | 4.29 | 3.68 |
60
+ | chirp3_3_short | 104 | 91.35 | 0.933 | 0.875 | 0.903 | 2.88 | 5.77 |
61
+ | chirp3_2 | 8,428 | 90.76 | 0.898 | 0.918 | 0.908 | 5.17 | 4.07 |
62
+ | human_convcollector_1 | 90 | 90.00 | 0.837 | 0.947 | 0.889 | 7.78 | 2.22 |
63
+ | orpheus_midfiller_1 | 140 | 87.86 | 0.859 | 0.873 | 0.866 | 6.43 | 5.71 |
64
+ | mundo_1 | 496 | 87.70 | 0.871 | 0.882 | 0.877 | 6.45 | 5.85 |
65
+
smart-turn-v3.0.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:07a133aba31e2d0b523f17f8c2e4e65efe6d8f685efd12ca4fe21ebf4e798991
3
+ size 8757193
smart-turn-v3.1-cpu.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb68d55c2d542ce79e44b12013bfd571e90df8594ab096d757198e851b0c6594
3
+ size 8679180
smart-turn-v3.1-gpu.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a32f7445d5076029472b6c9f7a71005df576ea19d5f929021200f535b962af84
3
+ size 32411198
smart-turn-v3.2-cpu.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2bb026316b14a660486a75b1733cd3fbab8c2fd0314dc9af7be49f8cca967e4f
3
+ size 8679182
smart-turn-v3.2-gpu.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ab8dc64b88713f90b571c15b714bd1330e6c883cad8763dacf65c9376dc539be
3
+ size 32411198