File size: 5,206 Bytes
16c8130 f766f81 16c8130 f766f81 16c8130 f766f81 16c8130 f766f81 16c8130 f766f81 16c8130 f766f81 16c8130 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | # Endpointing Model Benchmark Report
**Model:** `/data/smart-turn-v3.1-cpu.onnx`
**Generated:** 2026-01-07 17:38:46 UTC
## Accuracy Results
**Total Samples:** 31,527
**Unique Languages:** 🇸🇦 Arabic, 🇧🇩 Bengali, 🇩🇰 Danish, 🇩🇪 German, 🇬🇧 🇺🇸 English, 🇫🇮 Finnish, 🇫🇷 French, 🇮🇳 Hindi, 🇮🇩 Indonesian, 🇮🇹 Italian, 🇯🇵 Japanese, 🇰🇷 Korean, 🇮🇳 Marathi, 🇳🇱 Dutch, 🇳🇴 Norwegian, 🇵🇱 Polish, 🇵🇹 Portuguese, 🇷🇺 Russian, 🇪🇸 Spanish, 🇹🇷 Turkish, 🇺🇦 Ukrainian, 🇻🇳 Vietnamese, 🇨🇳 Chinese
**Unique Datasets:** chirp3_1, chirp3_2, chirp3_3_short, human_5, human_convcollector_1, liva_1, midcentury_1, mundo_1, orpheus_endfiller_1, orpheus_grammar_1, orpheus_midfiller_1, rime_2
### Overall Performance
| Metric | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
| :------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
| Overall | 31,527 | 90.13 | 0.883 | 0.924 | 0.903 | 6.08 | 3.79 |
### Performance by Language
| Language | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
| :------------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
| 🇯🇵 Japanese | 834 | 95.32 | 0.936 | 0.974 | 0.954 | 3.36 | 1.32 |
| 🇫🇷 French | 1,252 | 94.89 | 0.949 | 0.951 | 0.950 | 2.64 | 2.48 |
| 🇳🇱 Dutch | 1,398 | 94.71 | 0.943 | 0.956 | 0.949 | 3.00 | 2.29 |
| 🇩🇪 German | 1,322 | 94.02 | 0.928 | 0.954 | 0.941 | 3.71 | 2.27 |
| 🇵🇱 Polish | 974 | 93.43 | 0.910 | 0.957 | 0.933 | 4.52 | 2.05 |
| 🇰🇷 Korean | 889 | 93.03 | 0.944 | 0.914 | 0.929 | 2.70 | 4.27 |
| 🇵🇹 Portuguese | 1,398 | 92.85 | 0.946 | 0.904 | 0.925 | 2.50 | 4.65 |
| 🇹🇷 Turkish | 966 | 92.75 | 0.900 | 0.960 | 0.929 | 5.28 | 1.97 |
| 🇮🇩 Indonesian | 971 | 92.17 | 0.912 | 0.931 | 0.921 | 4.43 | 3.40 |
| 🇮🇹 Italian | 782 | 91.94 | 0.881 | 0.969 | 0.923 | 6.52 | 1.53 |
| 🇷🇺 Russian | 1,468 | 91.42 | 0.901 | 0.940 | 0.920 | 5.45 | 3.13 |
| 🇬🇧 🇺🇸 English | 7,820 | 90.66 | 0.885 | 0.930 | 0.907 | 5.92 | 3.41 |
| 🇩🇰 Danish | 779 | 90.63 | 0.876 | 0.949 | 0.911 | 6.80 | 2.57 |
| 🇮🇳 Hindi | 1,284 | 90.58 | 0.897 | 0.925 | 0.911 | 5.53 | 3.89 |
| 🇺🇦 Ukrainian | 929 | 89.99 | 0.869 | 0.927 | 0.897 | 6.57 | 3.44 |
| 🇳🇴 Norwegian | 1,014 | 88.95 | 0.855 | 0.944 | 0.897 | 8.19 | 2.86 |
| 🇫🇮 Finnish | 1,010 | 88.32 | 0.831 | 0.960 | 0.891 | 9.70 | 1.98 |
| 🇪🇸 Spanish | 1,783 | 87.49 | 0.854 | 0.898 | 0.875 | 7.52 | 4.99 |
| 🇸🇦 Arabic | 947 | 85.96 | 0.831 | 0.909 | 0.868 | 9.40 | 4.65 |
| 🇨🇳 Chinese | 929 | 85.25 | 0.833 | 0.888 | 0.860 | 9.04 | 5.71 |
| 🇮🇳 Marathi | 774 | 82.30 | 0.792 | 0.883 | 0.835 | 11.76 | 5.94 |
| 🇧🇩 Bengali | 1,000 | 80.30 | 0.774 | 0.845 | 0.808 | 12.10 | 7.60 |
| 🇻🇳 Vietnamese | 1,004 | 77.99 | 0.805 | 0.735 | 0.769 | 8.86 | 13.15 |
### Performance by Dataset
| Dataset | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
| :-------------------- | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
| orpheus_endfiller_1 | 181 | 95.58 | 1.000 | 0.913 | 0.955 | 0.00 | 4.42 |
| liva_1 | 3,831 | 93.45 | 0.929 | 0.942 | 0.935 | 3.63 | 2.92 |
| human_5 | 402 | 93.03 | 0.931 | 0.910 | 0.920 | 2.99 | 3.98 |
| rime_2 | 394 | 92.89 | 0.932 | 0.898 | 0.915 | 2.79 | 4.31 |
| chirp3_1 | 16,254 | 92.09 | 0.907 | 0.938 | 0.923 | 4.82 | 3.09 |
| orpheus_grammar_1 | 163 | 89.57 | 0.895 | 0.906 | 0.901 | 5.52 | 4.91 |
| orpheus_midfiller_1 | 140 | 89.29 | 0.875 | 0.889 | 0.882 | 5.71 | 5.00 |
| chirp3_2 | 8,428 | 86.24 | 0.838 | 0.897 | 0.866 | 8.64 | 5.13 |
| human_convcollector_1 | 90 | 85.56 | 0.791 | 0.895 | 0.840 | 10.00 | 4.44 |
| mundo_1 | 496 | 82.06 | 0.815 | 0.825 | 0.820 | 9.27 | 8.67 |
| midcentury_1 | 1,044 | 81.32 | 0.742 | 0.940 | 0.829 | 15.80 | 2.87 |
| chirp3_3_short | 104 | 78.85 | 0.842 | 0.667 | 0.744 | 5.77 | 15.38 |
|