File size: 5,202 Bytes
16c8130 f766f81 16c8130 f766f81 16c8130 f766f81 16c8130 f766f81 16c8130 f766f81 16c8130 f766f81 16c8130 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | # Endpointing Model Benchmark Report
**Model:** `/data/smart-turn-v3.0.onnx`
**Generated:** 2026-01-07 17:31:01 UTC
## Accuracy Results
**Total Samples:** 31,527
**Unique Languages:** 🇸🇦 Arabic, 🇧🇩 Bengali, 🇩🇰 Danish, 🇩🇪 German, 🇬🇧 🇺🇸 English, 🇫🇮 Finnish, 🇫🇷 French, 🇮🇳 Hindi, 🇮🇩 Indonesian, 🇮🇹 Italian, 🇯🇵 Japanese, 🇰🇷 Korean, 🇮🇳 Marathi, 🇳🇱 Dutch, 🇳🇴 Norwegian, 🇵🇱 Polish, 🇵🇹 Portuguese, 🇷🇺 Russian, 🇪🇸 Spanish, 🇹🇷 Turkish, 🇺🇦 Ukrainian, 🇻🇳 Vietnamese, 🇨🇳 Chinese
**Unique Datasets:** chirp3_1, chirp3_2, chirp3_3_short, human_5, human_convcollector_1, liva_1, midcentury_1, mundo_1, orpheus_endfiller_1, orpheus_grammar_1, orpheus_midfiller_1, rime_2
### Overall Performance
| Metric | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
| :------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
| Overall | 31,527 | 88.97 | 0.858 | 0.933 | 0.894 | 7.70 | 3.33 |
### Performance by Language
| Language | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
| :------------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
| 🇰🇷 Korean | 889 | 95.39 | 0.947 | 0.962 | 0.954 | 2.70 | 1.91 |
| 🇹🇷 Turkish | 966 | 95.34 | 0.927 | 0.983 | 0.954 | 3.83 | 0.83 |
| 🇩🇪 German | 1,322 | 95.23 | 0.928 | 0.980 | 0.954 | 3.78 | 0.98 |
| 🇵🇹 Portuguese | 1,398 | 94.49 | 0.927 | 0.962 | 0.944 | 3.65 | 1.86 |
| 🇯🇵 Japanese | 834 | 94.36 | 0.925 | 0.967 | 0.945 | 3.96 | 1.68 |
| 🇳🇱 Dutch | 1,398 | 94.28 | 0.925 | 0.968 | 0.946 | 4.08 | 1.65 |
| 🇫🇷 French | 1,252 | 94.25 | 0.945 | 0.942 | 0.944 | 2.80 | 2.96 |
| 🇮🇹 Italian | 782 | 93.22 | 0.898 | 0.974 | 0.935 | 5.50 | 1.28 |
| 🇵🇱 Polish | 974 | 93.02 | 0.904 | 0.955 | 0.929 | 4.83 | 2.16 |
| 🇷🇺 Russian | 1,468 | 92.03 | 0.891 | 0.966 | 0.927 | 6.20 | 1.77 |
| 🇺🇦 Ukrainian | 929 | 91.82 | 0.881 | 0.954 | 0.916 | 6.03 | 2.15 |
| 🇮🇩 Indonesian | 971 | 90.73 | 0.854 | 0.979 | 0.912 | 8.24 | 1.03 |
| 🇮🇳 Hindi | 1,284 | 89.95 | 0.856 | 0.970 | 0.910 | 8.49 | 1.56 |
| 🇩🇰 Danish | 779 | 89.73 | 0.841 | 0.982 | 0.906 | 9.37 | 0.90 |
| 🇳🇴 Norwegian | 1,014 | 89.64 | 0.869 | 0.938 | 0.903 | 7.20 | 3.16 |
| 🇨🇳 Chinese | 929 | 89.34 | 0.861 | 0.943 | 0.900 | 7.75 | 2.91 |
| 🇫🇮 Finnish | 1,010 | 87.52 | 0.813 | 0.974 | 0.886 | 11.19 | 1.29 |
| 🇸🇦 Arabic | 947 | 86.17 | 0.810 | 0.950 | 0.875 | 11.30 | 2.53 |
| 🇬🇧 🇺🇸 English | 7,820 | 85.49 | 0.836 | 0.876 | 0.855 | 8.41 | 6.10 |
| 🇪🇸 Spanish | 1,783 | 84.46 | 0.848 | 0.831 | 0.839 | 7.29 | 8.24 |
| 🇮🇳 Marathi | 774 | 81.01 | 0.751 | 0.936 | 0.834 | 15.76 | 3.23 |
| 🇻🇳 Vietnamese | 1,004 | 79.58 | 0.721 | 0.960 | 0.824 | 18.43 | 1.99 |
| 🇧🇩 Bengali | 1,000 | 78.70 | 0.717 | 0.935 | 0.811 | 18.10 | 3.20 |
### Performance by Dataset
| Dataset | Sample Count | Accuracy (%) | Precision | Recall | F1 | FPR (%) | FNR (%) |
| :-------------------- | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
| rime_2 | 394 | 95.94 | 0.981 | 0.922 | 0.951 | 0.76 | 3.30 |
| orpheus_endfiller_1 | 181 | 95.03 | 1.000 | 0.902 | 0.949 | 0.00 | 4.97 |
| human_5 | 402 | 94.03 | 0.948 | 0.916 | 0.931 | 2.24 | 3.73 |
| chirp3_1 | 16,254 | 92.84 | 0.903 | 0.961 | 0.931 | 5.19 | 1.96 |
| human_convcollector_1 | 90 | 91.11 | 0.895 | 0.895 | 0.895 | 4.44 | 4.44 |
| orpheus_midfiller_1 | 140 | 90.00 | 0.877 | 0.905 | 0.891 | 5.71 | 4.29 |
| orpheus_grammar_1 | 163 | 87.12 | 0.848 | 0.918 | 0.881 | 8.59 | 4.29 |
| chirp3_2 | 8,428 | 86.08 | 0.802 | 0.956 | 0.872 | 11.75 | 2.17 |
| liva_1 | 3,831 | 83.74 | 0.839 | 0.838 | 0.838 | 8.09 | 8.17 |
| midcentury_1 | 1,044 | 77.39 | 0.704 | 0.917 | 0.797 | 18.58 | 4.02 |
| chirp3_3_short | 104 | 74.04 | 0.818 | 0.562 | 0.667 | 5.77 | 20.19 |
| mundo_1 | 496 | 67.34 | 0.741 | 0.524 | 0.614 | 9.07 | 23.59 |
|