# Endpointing Model Benchmark Report

**Model:** `/data/smart-turn-v3.1-gpu.onnx`

**Generated:** 2026-01-07 17:45:59 UTC

## Accuracy Results

**Total Samples:** 31,527

**Unique Languages:** 🇸🇦 Arabic, 🇧🇩 Bengali, 🇩🇰 Danish, 🇩🇪 German, 🇬🇧 🇺🇸 English, 🇫🇮 Finnish, 🇫🇷 French, 🇮🇳 Hindi, 🇮🇩 Indonesian, 🇮🇹 Italian, 🇯🇵 Japanese, 🇰🇷 Korean, 🇮🇳 Marathi, 🇳🇱 Dutch, 🇳🇴 Norwegian, 🇵🇱 Polish, 🇵🇹 Portuguese, 🇷🇺 Russian, 🇪🇸 Spanish, 🇹🇷 Turkish, 🇺🇦 Ukrainian, 🇻🇳 Vietnamese, 🇨🇳 Chinese

**Unique Datasets:** chirp3_1, chirp3_2, chirp3_3_short, human_5, human_convcollector_1, liva_1, midcentury_1, mundo_1, orpheus_endfiller_1, orpheus_grammar_1, orpheus_midfiller_1, rime_2

### Overall Performance

| Metric  | Sample Count | Accuracy (%) | Precision | Recall |    F1 | FPR (%) | FNR (%) |
| :------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
| Overall |       31,527 |        91.64 |     0.894 |  0.944 | 0.918 |    5.55 |    2.81 |

### Performance by Language

| Language      | Sample Count | Accuracy (%) | Precision | Recall |    F1 | FPR (%) | FNR (%) |
| :------------ | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
| 🇯🇵 Japanese   |          834 |        95.68 |     0.944 |  0.971 | 0.958 |    2.88 |    1.44 |
| 🇳🇱 Dutch      |        1,398 |        95.42 |     0.950 |  0.963 | 0.956 |    2.65 |    1.93 |
| 🇹🇷 Turkish    |          966 |        95.34 |     0.935 |  0.973 | 0.954 |    3.31 |    1.35 |
| 🇫🇷 French     |        1,252 |        95.29 |     0.950 |  0.958 | 0.954 |    2.56 |    2.16 |
| 🇰🇷 Korean     |          889 |        95.16 |     0.965 |  0.937 | 0.951 |    1.69 |    3.15 |
| 🇩🇪 German     |        1,322 |        95.16 |     0.936 |  0.970 | 0.952 |    3.33 |    1.51 |
| 🇵🇹 Portuguese |        1,398 |        94.85 |     0.942 |  0.953 | 0.947 |    2.86 |    2.29 |
| 🇮🇹 Italian    |          782 |        94.50 |     0.922 |  0.972 | 0.946 |    4.09 |    1.41 |
| 🇵🇱 Polish     |          974 |        94.35 |     0.921 |  0.963 | 0.942 |    3.90 |    1.75 |
| 🇮🇩 Indonesian |          971 |        93.10 |     0.905 |  0.960 | 0.932 |    4.94 |    1.96 |
| 🇷🇺 Russian    |        1,468 |        92.64 |     0.911 |  0.953 | 0.932 |    4.90 |    2.45 |
| 🇮🇳 Hindi      |        1,284 |        92.52 |     0.919 |  0.939 | 0.929 |    4.28 |    3.19 |
| 🇺🇦 Ukrainian  |          929 |        92.03 |     0.900 |  0.933 | 0.917 |    4.84 |    3.12 |
| 🇬🇧 🇺🇸 English |        7,820 |        91.94 |     0.889 |  0.954 | 0.921 |    5.82 |    2.24 |
| 🇩🇰 Danish     |          779 |        91.14 |     0.880 |  0.954 | 0.916 |    6.55 |    2.31 |
| 🇫🇮 Finnish    |        1,010 |        90.50 |     0.859 |  0.968 | 0.910 |    7.92 |    1.58 |
| 🇳🇴 Norwegian  |        1,014 |        89.84 |     0.865 |  0.950 | 0.905 |    7.59 |    2.56 |
| 🇪🇸 Spanish    |        1,783 |        89.62 |     0.871 |  0.924 | 0.897 |    6.67 |    3.70 |
| 🇨🇳 Chinese    |          929 |        88.37 |     0.850 |  0.937 | 0.891 |    8.40 |    3.23 |
| 🇸🇦 Arabic     |          947 |        87.01 |     0.838 |  0.923 | 0.878 |    9.08 |    3.91 |
| 🇮🇳 Marathi    |          774 |        84.88 |     0.833 |  0.878 | 0.855 |    8.91 |    6.20 |
| 🇧🇩 Bengali    |        1,000 |        81.20 |     0.801 |  0.820 | 0.810 |   10.00 |    8.80 |
| 🇻🇳 Vietnamese |        1,004 |        81.08 |     0.780 |  0.862 | 0.819 |   12.05 |    6.87 |

### Performance by Dataset

| Dataset               | Sample Count | Accuracy (%) | Precision | Recall |    F1 | FPR (%) | FNR (%) |
| :-------------------- | -----------: | -----------: | --------: | -----: | ----: | ------: | ------: |
| orpheus_endfiller_1   |          181 |        97.24 |     1.000 |  0.946 | 0.972 |    0.00 |    2.76 |
| rime_2                |          394 |        97.21 |     0.959 |  0.976 | 0.967 |    1.78 |    1.02 |
| human_5               |          402 |        95.02 |     0.939 |  0.949 | 0.944 |    2.74 |    2.24 |
| liva_1                |        3,831 |        94.23 |     0.929 |  0.959 | 0.944 |    3.68 |    2.09 |
| chirp3_1              |       16,254 |        93.53 |     0.919 |  0.955 | 0.937 |    4.22 |    2.25 |
| orpheus_grammar_1     |          163 |        89.57 |     0.878 |  0.929 | 0.903 |    6.75 |    3.68 |
| orpheus_midfiller_1   |          140 |        89.29 |     0.853 |  0.921 | 0.885 |    7.14 |    3.57 |
| chirp3_2              |        8,428 |        87.81 |     0.850 |  0.916 | 0.882 |    8.03 |    4.15 |
| chirp3_3_short        |          104 |        85.58 |     0.867 |  0.812 | 0.839 |    5.77 |    8.65 |
| mundo_1               |          496 |        84.68 |     0.840 |  0.854 | 0.847 |    8.06 |    7.26 |
| human_convcollector_1 |           90 |        84.44 |     0.761 |  0.921 | 0.833 |   12.22 |    3.33 |
| midcentury_1          |        1,044 |        84.39 |     0.766 |  0.974 | 0.858 |   14.37 |    1.25 |