| Dataset | Run | Req | Output tok/s | Speedup | Latency mean/p95 ms | TTFT mean/p95 ms | ITL mean/p95 ms | TPOT mean/p95 ms | Accept rate | Accept p1/p2/p3 | Accept len |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| math500 | baseline | 500 | 327.25 | 1.000x | 1549.88/1901.40 | 147.72/277.21 | 11.12/13.80 | 11.12/13.80 | — | -/-/- | — |
| math500 | dspark_k3 | 500 | 424.44 | 1.297x | 1195.68/1773.15 | 143.30/297.68 | 8.35/12.35 | 8.35/12.36 | 41.55% | 63.32%/38.55%/22.77% | 2.246 |
| math500 | dspark_k7 | 500 | 394.37 | 1.205x | 1286.87/1958.87 | 169.23/310.46 | 8.87/13.72 | 8.87/13.72 | 20.28% | 61.78%/36.53%/20.61% | 2.419 |
| gsm8k | baseline | 1319 | 344.96 | 1.000x | 1479.66/1803.74 | 168.78/336.33 | 10.34/12.40 | 10.34/12.41 | — | -/-/- | — |
| gsm8k | dspark_k3 | 1319 | 376.84 | 1.092x | 1352.79/1901.42 | 146.36/300.51 | 9.52/13.59 | 9.52/13.59 | 27.04% | 48.59%/22.00%/10.53% | 1.811 |
| gsm8k | dspark_k7 | 1319 | 361.79 | 1.049x | 1409.89/2024.30 | 149.38/303.52 | 9.94/14.45 | 9.94/14.45 | 12.44% | 47.59%/21.32%/10.22% | 1.871 |
| humaneval | baseline | 164 | 265.07 | 1.000x | 1027.53/2018.19 | 140.52/278.59 | 13.19/19.83 | 13.19/19.84 | — | -/-/- | — |
| humaneval | dspark_k3 | 164 | 371.59 | 1.402x | 731.68/1438.71 | 153.84/371.52 | 9.22/16.54 | 9.22/16.55 | 57.23% | 75.49%/55.07%/41.14% | 2.717 |
| humaneval | dspark_k7 | 164 | 388.96 | 1.467x | 695.93/1258.20 | 170.66/440.41 | 9.07/18.30 | 9.07/18.30 | 33.28% | 73.49%/54.04%/38.35% | 3.329 |
| mbpp | baseline | 500 | 345.00 | 1.000x | 1426.25/1792.68 | 146.01/278.13 | 10.54/12.80 | 10.55/12.80 | — | -/-/- | — |
| mbpp | dspark_k3 | 500 | 478.08 | 1.386x | 1031.81/1450.64 | 149.58/290.25 | 7.49/10.49 | 7.49/10.50 | 51.15% | 70.32%/50.12%/33.02% | 2.535 |
| mbpp | dspark_k7 | 500 | 482.65 | 1.399x | 1016.76/1519.82 | 157.02/310.51 | 7.46/11.04 | 7.47/11.13 | 27.60% | 67.48%/46.77%/29.47% | 2.932 |
| mt_bench | baseline | 160 | 359.97 | 1.000x | 1379.88/1778.27 | 230.35/469.93 | 9.20/12.18 | 9.20/12.19 | — | -/-/- | — |
| mt_bench | dspark_k3 | 160 | 395.71 | 1.099x | 1267.36/1889.96 | 149.93/296.09 | 8.95/13.28 | 8.96/13.30 | 32.00% | 54.34%/27.67%/14.00% | 1.960 |
| mt_bench | dspark_k7 | 160 | 356.55 | 0.991x | 1404.46/2271.47 | 182.32/363.33 | 9.92/17.31 | 9.93/17.33 | 14.60% | 51.91%/25.81%/12.52% | 2.022 |
| ultrachat | baseline | 5000 | 169.47 | 1.000x | 2999.21/3652.68 | 191.45/319.01 | 22.26/26.93 | 22.26/26.93 | — | -/-/- | — |
| ultrachat | dspark_k3 | 5000 | 231.21 | 1.364x | 2200.76/2860.88 | 203.06/327.66 | 15.82/20.66 | 15.82/20.66 | 30.23% | 53.29%/25.70%/11.69% | 1.907 |
| ultrachat | dspark_k7 | 5000 | 230.66 | 1.361x | 2205.89/2923.48 | 208.65/335.64 | 15.81/21.20 | 15.82/21.20 | 14.16% | 53.16%/25.40%/11.49% | 1.991 |