| Dataset | Run | Req | Output tok/s | Speedup | Latency mean/p95 ms | TTFT mean/p95 ms | ITL mean/p95 ms | TPOT mean/p95 ms | Accept rate | Accept p1/p2/p3 | Accept len |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| math500 | baseline | 500 | 123.60 | 1.000x | 1029.65/1183.88 | 137.18/254.18 | 7.07/8.15 | 7.07/8.15 | — | -/-/- | — |
| math500 | dspark_k3 | 500 | 152.19 | 1.231x | 837.38/1146.01 | 119.19/213.45 | 5.68/8.04 | 5.68/8.04 | 41.56% | 63.26%/38.63%/22.80% | 2.247 |
| math500 | dspark_k7 | 500 | 149.95 | 1.213x | 849.09/1242.33 | 129.85/256.74 | 5.69/8.78 | 5.69/8.78 | 20.32% | 62.06%/36.62%/20.48% | 2.423 |
| gsm8k | baseline | 1319 | 127.30 | 1.000x | 1003.43/1147.13 | 121.83/253.88 | 6.95/7.24 | 6.96/7.24 | — | -/-/- | — |
| gsm8k | dspark_k3 | 1319 | 128.83 | 1.012x | 990.98/1346.59 | 111.31/207.46 | 6.94/9.58 | 6.94/9.58 | 27.13% | 48.78%/22.05%/10.56% | 1.814 |
| gsm8k | dspark_k7 | 1319 | 121.81 | 0.957x | 1048.34/1480.12 | 127.43/265.25 | 7.26/10.53 | 7.27/10.53 | 12.43% | 47.85%/21.34%/10.03% | 1.870 |
| humaneval | baseline | 164 | 112.12 | 1.000x | 630.40/1166.23 | 107.50/235.70 | 7.30/8.49 | 7.30/8.50 | — | -/-/- | — |
| humaneval | dspark_k3 | 164 | 162.20 | 1.447x | 428.57/818.54 | 100.60/181.40 | 5.12/7.48 | 5.13/7.48 | 57.68% | 75.54%/55.69%/41.82% | 2.730 |
| humaneval | dspark_k7 | 164 | 149.24 | 1.331x | 444.15/816.37 | 160.06/265.69 | 4.94/8.73 | 4.95/8.75 | 33.09% | 73.07%/53.81%/38.30% | 3.317 |
| mbpp | baseline | 500 | 130.38 | 1.000x | 942.76/1136.07 | 128.44/256.76 | 6.67/7.12 | 6.67/7.12 | — | -/-/- | — |
| mbpp | dspark_k3 | 500 | 175.87 | 1.349x | 700.90/919.01 | 108.79/204.45 | 5.01/6.63 | 5.01/6.63 | 50.96% | 70.10%/49.94%/32.84% | 2.529 |
| mbpp | dspark_k7 | 500 | 175.67 | 1.347x | 696.66/978.64 | 144.75/269.57 | 4.79/6.83 | 4.80/6.83 | 27.43% | 67.37%/46.24%/29.18% | 2.920 |
| mt_bench | baseline | 160 | 127.47 | 1.000x | 985.54/1160.10 | 135.88/258.65 | 6.80/7.93 | 6.81/7.93 | — | -/-/- | — |
| mt_bench | dspark_k3 | 160 | 144.01 | 1.130x | 872.29/1313.48 | 97.45/185.92 | 6.34/10.21 | 6.34/10.33 | 32.28% | 54.41%/28.24%/14.18% | 1.968 |
| mt_bench | dspark_k7 | 160 | 132.15 | 1.037x | 950.60/1560.65 | 132.69/258.32 | 6.66/11.56 | 6.66/11.56 | 14.79% | 52.76%/25.79%/12.72% | 2.035 |
| ultrachat | baseline | 5000 | 66.23 | 1.000x | 1920.77/3165.01 | 134.13/226.60 | 14.15/23.34 | 14.15/23.34 | — | -/-/- | — |
| ultrachat | dspark_k3 | 5000 | 83.93 | 1.267x | 1515.15/2205.47 | 154.38/250.98 | 10.78/15.62 | 10.78/15.62 | 30.25% | 53.35%/25.69%/11.70% | 1.907 |
| ultrachat | dspark_k7 | 5000 | 83.77 | 1.265x | 1518.37/2203.91 | 165.57/262.97 | 10.72/15.63 | 10.72/15.63 | 14.19% | 53.12%/25.50%/11.52% | 1.993 |