| Dataset | Run | Req | Output tok/s | Speedup | Latency mean/p95 ms | TTFT mean/p95 ms | ITL mean/p95 ms | TPOT mean/p95 ms | Accept rate | Accept p1/p2/p3 | Accept len |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| math500 | baseline | 500 | 113.45 | 1.000x | 1121.85/1314.59 | 169.94/335.47 | 7.54/8.68 | 7.54/8.69 | — | -/-/- | — |
| math500 | dflash_k3 | 500 | 64.68 | 0.570x | 1970.50/2394.33 | 176.98/348.85 | 14.18/17.33 | 14.18/17.33 | 6.32% | 16.82%/1.89%/0.26% | 1.190 |
| math500 | dflash_k7 | 500 | 74.16 | 0.654x | 1716.92/2122.72 | 138.95/232.84 | 12.49/15.42 | 12.49/15.42 | 2.73% | 16.89%/1.92%/0.26% | 1.191 |
| math500 | dflare_k3 | 500 | 93.74 | 0.826x | 1359.56/1919.75 | 150.99/254.30 | 9.56/13.64 | 9.56/13.64 | 39.18% | 64.52%/35.08%/17.94% | 2.175 |
| math500 | dflare_k7 | 500 | 97.45 | 0.859x | 1306.61/1831.94 | 149.97/241.51 | 9.16/13.16 | 9.17/13.16 | 18.55% | 64.82%/35.30%/17.18% | 2.298 |
| math500 | dspark_k3 | 500 | 131.01 | 1.155x | 972.82/1383.58 | 139.16/243.83 | 6.59/9.67 | 6.59/9.67 | 44.80% | 65.41%/42.19%/26.82% | 2.344 |
| math500 | dspark_k7 | 500 | 136.48 | 1.203x | 932.87/1415.98 | 142.46/239.27 | 6.26/10.08 | 6.26/10.08 | 22.84% | 64.00%/39.61%/24.04% | 2.599 |
| gsm8k | baseline | 1319 | 115.56 | 1.000x | 1105.36/1276.34 | 164.78/332.90 | 7.42/7.93 | 7.42/7.93 | — | -/-/- | — |
| gsm8k | dflash_k3 | 1319 | 65.68 | 0.568x | 1943.86/2365.31 | 178.53/355.62 | 13.93/16.93 | 13.93/16.93 | 6.84% | 18.35%/1.95%/0.21% | 1.205 |
| gsm8k | dflash_k7 | 1319 | 73.26 | 0.634x | 1743.27/2099.80 | 152.56/246.42 | 12.55/15.04 | 12.55/15.04 | 2.75% | 17.19%/1.83%/0.20% | 1.192 |
| gsm8k | dflare_k3 | 1319 | 80.95 | 0.701x | 1577.24/2166.57 | 150.18/254.45 | 11.26/15.61 | 11.26/15.61 | 27.89% | 53.39%/21.58%/8.69% | 1.837 |
| gsm8k | dflare_k7 | 1319 | 81.92 | 0.709x | 1558.94/2191.44 | 152.78/255.01 | 11.09/15.86 | 11.09/15.86 | 12.80% | 54.17%/21.95%/9.32% | 1.896 |
| gsm8k | dspark_k3 | 1319 | 111.95 | 0.969x | 1140.44/1627.45 | 137.90/232.76 | 7.91/11.64 | 7.91/11.64 | 31.19% | 53.50%/25.78%/14.30% | 1.936 |
| gsm8k | dspark_k7 | 1319 | 111.09 | 0.961x | 1149.49/1690.77 | 142.42/238.71 | 7.94/11.94 | 7.94/11.94 | 14.63% | 53.36%/25.01%/13.27% | 2.024 |
| humaneval | baseline | 164 | 93.66 | 1.000x | 754.68/1367.14 | 189.28/342.50 | 8.11/9.87 | 8.11/9.87 | — | -/-/- | — |
| humaneval | dflash_k3 | 164 | 58.98 | 0.630x | 1178.81/2207.20 | 162.99/323.05 | 15.05/20.25 | 15.06/20.27 | 5.99% | 16.43%/1.45%/0.10% | 1.180 |
| humaneval | dflash_k7 | 164 | 64.81 | 0.692x | 1022.86/2083.64 | 157.66/238.51 | 13.24/16.24 | 13.24/16.24 | 2.71% | 17.46%/1.41%/0.08% | 1.189 |
| humaneval | dflare_k3 | 164 | 99.28 | 1.060x | 700.23/1298.74 | 163.54/255.47 | 8.18/11.82 | 8.19/11.83 | 58.63% | 77.59%/56.43%/41.86% | 2.759 |
| humaneval | dflare_k7 | 164 | 114.43 | 1.222x | 579.29/1124.12 | 153.89/249.90 | 7.29/11.29 | 7.29/11.30 | 32.60% | 76.90%/55.26%/40.09% | 3.282 |
| humaneval | dspark_k3 | 164 | 130.99 | 1.399x | 530.74/905.70 | 146.04/236.92 | 6.10/8.98 | 6.11/8.99 | 60.43% | 77.12%/58.94%/45.24% | 2.813 |
| humaneval | dspark_k7 | 164 | 145.41 | 1.553x | 455.86/852.21 | 143.63/241.35 | 5.51/9.66 | 5.51/9.66 | 34.97% | 73.45%/55.20%/39.37% | 3.448 |
| mbpp | baseline | 500 | 117.61 | 1.000x | 1045.16/1252.24 | 170.20/343.01 | 7.19/7.64 | 7.19/7.64 | — | -/-/- | — |
| mbpp | dflash_k3 | 500 | 67.35 | 0.573x | 1830.21/2291.14 | 181.88/374.19 | 13.37/16.29 | 13.37/16.29 | 6.52% | 18.39%/1.10%/0.06% | 1.196 |
| mbpp | dflash_k7 | 500 | 74.82 | 0.636x | 1635.87/2016.28 | 152.67/244.47 | 12.10/14.41 | 12.10/14.41 | 2.84% | 18.66%/1.14%/0.06% | 1.199 |
| mbpp | dflare_k3 | 500 | 103.43 | 0.879x | 1191.91/1663.97 | 151.14/261.67 | 8.62/12.05 | 8.62/12.05 | 46.96% | 68.67%/44.38%/27.84% | 2.409 |
| mbpp | dflare_k7 | 500 | 115.48 | 0.982x | 1059.83/1511.53 | 149.30/254.47 | 7.82/11.13 | 7.83/11.13 | 24.50% | 67.47%/42.50%/25.01% | 2.715 |
| mbpp | dspark_k3 | 500 | 146.73 | 1.248x | 840.11/1144.96 | 133.81/238.02 | 5.95/8.07 | 5.95/8.07 | 52.37% | 71.03%/51.06%/35.03% | 2.571 |
| mbpp | dspark_k7 | 500 | 158.24 | 1.346x | 773.38/1069.82 | 144.88/240.50 | 5.46/7.70 | 5.46/7.70 | 28.85% | 68.24%/46.95%/30.63% | 3.020 |
| mt_bench | baseline | 160 | 116.89 | 1.000x | 1074.68/1319.75 | 160.69/327.47 | 7.31/8.34 | 7.31/8.34 | — | -/-/- | — |
| mt_bench | dflash_k3 | 160 | 65.85 | 0.563x | 1907.75/2463.49 | 175.25/376.67 | 13.83/17.45 | 13.84/17.45 | 6.04% | 16.21%/1.76%/0.15% | 1.181 |
| mt_bench | dflash_k7 | 160 | 73.27 | 0.627x | 1714.64/2135.27 | 139.66/237.62 | 12.69/15.55 | 12.69/15.55 | 2.35% | 14.88%/1.46%/0.09% | 1.164 |
| mt_bench | dflare_k3 | 160 | 85.55 | 0.732x | 1468.51/2127.07 | 159.73/265.74 | 10.65/15.71 | 10.66/15.71 | 32.31% | 57.74%/27.25%/11.94% | 1.969 |
| mt_bench | dflare_k7 | 160 | 89.12 | 0.762x | 1409.59/2124.10 | 146.91/259.67 | 10.23/15.34 | 10.24/15.34 | 14.63% | 57.43%/27.01%/11.06% | 2.024 |
| mt_bench | dspark_k3 | 160 | 121.09 | 1.036x | 1037.41/1565.51 | 141.77/240.41 | 7.31/11.71 | 7.32/11.71 | 37.06% | 59.00%/32.88%/19.30% | 2.112 |
| mt_bench | dspark_k7 | 160 | 122.24 | 1.046x | 1027.68/1596.33 | 141.25/255.56 | 7.28/11.79 | 7.29/11.79 | 17.42% | 56.41%/30.24%/17.08% | 2.220 |
| ultrachat | baseline | 5000 | 62.83 | 1.000x | 2024.82/3256.73 | 166.96/289.88 | 14.71/23.86 | 14.71/23.87 | — | -/-/- | — |
| ultrachat | dflash_k3 | 5000 | 52.06 | 0.829x | 2442.49/3377.26 | 168.36/252.91 | 18.01/24.71 | 18.01/24.71 | 7.08% | 18.36%/2.59%/0.30% | 1.212 |
| ultrachat | dflash_k7 | 5000 | 51.13 | 0.814x | 2487.86/3443.12 | 172.78/258.77 | 18.33/25.19 | 18.33/25.19 | 3.06% | 18.49%/2.58%/0.32% | 1.215 |
| ultrachat | dflare_k3 | 5000 | 62.40 | 0.993x | 2037.94/2756.50 | 185.96/279.43 | 14.67/19.80 | 14.67/19.80 | 31.13% | 57.22%/26.06%/10.11% | 1.934 |
| ultrachat | dflare_k7 | 5000 | 66.03 | 1.051x | 1925.98/2635.48 | 180.71/272.82 | 13.82/18.90 | 13.82/18.90 | 14.39% | 57.70%/26.66%/10.39% | 2.008 |
| ultrachat | dspark_k3 | 5000 | 78.10 | 1.243x | 1628.21/2287.42 | 171.98/258.65 | 11.54/16.24 | 11.54/16.25 | 32.41% | 55.57%/28.18%/13.47% | 1.972 |
| ultrachat | dspark_k7 | 5000 | 78.70 | 1.253x | 1616.26/2290.96 | 176.72/263.78 | 11.40/16.28 | 11.40/16.28 | 15.45% | 55.33%/28.03%/13.31% | 2.081 |