# Endpointing Model Benchmark Report

**Model:** `/data/smart-turn-v3.1-cpu.onnx`

**Generated:** 2026-01-07 17:38:46 UTC

## Accuracy Results

**Total Samples:** 31,527

**Unique Languages:** 🇸🇦 Arabic, 🇧🇩 Bengali, 🇩🇰 Danish, 🇩🇪 German, 🇬🇧 🇺🇸 English, 🇫🇮 Finnish, 🇫🇷 French, 🇮🇳 Hindi, 🇮🇩 Indonesian, 🇮🇹 Italian, 🇯🇵 Japanese, 🇰🇷 Korean, 🇮🇳 Marathi, 🇳🇱 Dutch, 🇳🇴 Norwegian, 🇵🇱 Polish, 🇵🇹 Portuguese, 🇷🇺 Russian, 🇪🇸 Spanish, 🇹🇷 Turkish, 🇺🇦 Ukrainian, 🇻🇳 Vietnamese, 🇨🇳 Chinese

**Unique Datasets:** chirp3_1, chirp3_2, chirp3_3_short, human_5, human_convcollector_1, liva_1, midcentury_1, mundo_1, orpheus_endfiller_1, orpheus_grammar_1, orpheus_midfiller_1, rime_2

### Overall Performance

| Metric  | Sample Count | Accuracy (%) | Precision | Recall |    F1 | FPR (%) | FNR (%) |
| :------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
| Overall |       31,527 |        90.13 |     0.883 |  0.924 | 0.903 |    6.08 |    3.79 |

### Performance by Language

| Language      | Sample Count | Accuracy (%) | Precision | Recall |    F1 | FPR (%) | FNR (%) |
| :------------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
| 🇯🇵 Japanese   |          834 |        95.32 |     0.936 |  0.974 | 0.954 |    3.36 |    1.32 |
| 🇫🇷 French     |        1,252 |        94.89 |     0.949 |  0.951 | 0.950 |    2.64 |    2.48 |
| 🇳🇱 Dutch      |        1,398 |        94.71 |     0.943 |  0.956 | 0.949 |    3.00 |    2.29 |
| 🇩🇪 German     |        1,322 |        94.02 |     0.928 |  0.954 | 0.941 |    3.71 |    2.27 |
| 🇵🇱 Polish     |          974 |        93.43 |     0.910 |  0.957 | 0.933 |    4.52 |    2.05 |
| 🇰🇷 Korean     |          889 |        93.03 |     0.944 |  0.914 | 0.929 |    2.70 |    4.27 |
| 🇵🇹 Portuguese |        1,398 |        92.85 |     0.946 |  0.904 | 0.925 |    2.50 |    4.65 |
| 🇹🇷 Turkish    |          966 |        92.75 |     0.900 |  0.960 | 0.929 |    5.28 |    1.97 |
| 🇮🇩 Indonesian |          971 |        92.17 |     0.912 |  0.931 | 0.921 |    4.43 |    3.40 |
| 🇮🇹 Italian    |          782 |        91.94 |     0.881 |  0.969 | 0.923 |    6.52 |    1.53 |
| 🇷🇺 Russian    |        1,468 |        91.42 |     0.901 |  0.940 | 0.920 |    5.45 |    3.13 |
| 🇬🇧 🇺🇸 English |        7,820 |        90.66 |     0.885 |  0.930 | 0.907 |    5.92 |    3.41 |
| 🇩🇰 Danish     |          779 |        90.63 |     0.876 |  0.949 | 0.911 |    6.80 |    2.57 |
| 🇮🇳 Hindi      |        1,284 |        90.58 |     0.897 |  0.925 | 0.911 |    5.53 |    3.89 |
| 🇺🇦 Ukrainian  |          929 |        89.99 |     0.869 |  0.927 | 0.897 |    6.57 |    3.44 |
| 🇳🇴 Norwegian  |        1,014 |        88.95 |     0.855 |  0.944 | 0.897 |    8.19 |    2.86 |
| 🇫🇮 Finnish    |        1,010 |        88.32 |     0.831 |  0.960 | 0.891 |    9.70 |    1.98 |
| 🇪🇸 Spanish    |        1,783 |        87.49 |     0.854 |  0.898 | 0.875 |    7.52 |    4.99 |
| 🇸🇦 Arabic     |          947 |        85.96 |     0.831 |  0.909 | 0.868 |    9.40 |    4.65 |
| 🇨🇳 Chinese    |          929 |        85.25 |     0.833 |  0.888 | 0.860 |    9.04 |    5.71 |
| 🇮🇳 Marathi    |          774 |        82.30 |     0.792 |  0.883 | 0.835 |   11.76 |    5.94 |
| 🇧🇩 Bengali    |        1,000 |        80.30 |     0.774 |  0.845 | 0.808 |   12.10 |    7.60 |
| 🇻🇳 Vietnamese |        1,004 |        77.99 |     0.805 |  0.735 | 0.769 |    8.86 |   13.15 |

### Performance by Dataset

| Dataset               | Sample Count | Accuracy (%) | Precision | Recall |    F1 | FPR (%) | FNR (%) |
| :-------------------- | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
| orpheus_endfiller_1   |          181 |        95.58 |     1.000 |  0.913 | 0.955 |    0.00 |    4.42 |
| liva_1                |        3,831 |        93.45 |     0.929 |  0.942 | 0.935 |    3.63 |    2.92 |
| human_5               |          402 |        93.03 |     0.931 |  0.910 | 0.920 |    2.99 |    3.98 |
| rime_2                |          394 |        92.89 |     0.932 |  0.898 | 0.915 |    2.79 |    4.31 |
| chirp3_1              |       16,254 |        92.09 |     0.907 |  0.938 | 0.923 |    4.82 |    3.09 |
| orpheus_grammar_1     |          163 |        89.57 |     0.895 |  0.906 | 0.901 |    5.52 |    4.91 |
| orpheus_midfiller_1   |          140 |        89.29 |     0.875 |  0.889 | 0.882 |    5.71 |    5.00 |
| chirp3_2              |        8,428 |        86.24 |     0.838 |  0.897 | 0.866 |    8.64 |    5.13 |
| human_convcollector_1 |           90 |        85.56 |     0.791 |  0.895 | 0.840 |   10.00 |    4.44 |
| mundo_1               |          496 |        82.06 |     0.815 |  0.825 | 0.820 |    9.27 |    8.67 |
| midcentury_1          |        1,044 |        81.32 |     0.742 |  0.940 | 0.829 |   15.80 |    2.87 |
| chirp3_3_short        |          104 |        78.85 |     0.842 |  0.667 | 0.744 |    5.77 |   15.38 |