File size: 5,122 Bytes
3b71a39 2261b3c 3b71a39 2261b3c 3b71a39 2261b3c 3b71a39 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 | ---
license: mit
library_name: transformers
pipeline_tag: text-generation
base_model:
- zai-org/GLM-5.2-FP8
datasets:
- JessieWei/GLM-5.2-FP8-nemotron-codealpaca
tags:
- dflash
- speculative-decoding
- draft-model
- block-diffusion
- glm
---
# GLM-5.2-FP8-DFlash
[Paper](https://arxiv.org/abs/2602.06036) | [DFlash GitHub](https://github.com/z-lab/dflash) | [SpecForge](https://github.com/sgl-project/SpecForge)
DFlash block-diffusion speculative-decoding drafter for
[GLM-5.2-FP8](https://huggingface.co/zai-org/GLM-5.2-FP8) (743B MoE, 39B active).
Standard DFlash method (no extensions), trained with
[SpecForge](https://github.com/sgl-project/SpecForge) on the paper-specified data
recipe: ~800K samples of Nemotron-Post-Training-v2 + CodeAlpaca (code / math / chat),
all responses regenerated by GLM-5.2-FP8, 6 epochs — directly comparable to the DFlash
paper and the z-lab drafter series.
## Quick Start (SGLang)
```bash
python -m sglang.launch_server \
--model-path zai-org/GLM-5.2-FP8 \
--speculative-algorithm DFLASH \
--speculative-draft-model-path UCloud-org/GLM-5.2-FP8-DFlash \
--speculative-num-draft-tokens 16 \
--tp-size 8 \
--trust-remote-code
```
vLLM v0.20.1+ has native DFlash support and reads this checkpoint directly,
no conversion needed. Not yet runtime-verified on our infrastructure; the
results below were produced via SGLang.
## Evaluation
<!-- Sampling protocol (t=1.0/top_p 0.95) and per-position acceptance table
to be added in v1.1. -->
### Mean accepted length & end-to-end speedup
Measured on live SGLang serving (concurrency 1, greedy decoding unless noted).
| Benchmark | AL (built-in MTP) | AL (DFlash) | DFlash throughput (tok/s) | Speedup vs vanilla | vs built-in MTP |
|-----------|-------------------|-------------|---------------------------|--------------------|-----------------|
| gsm8k | 4.01 | 4.44 | 236 | 2.22x | 1.51x |
| humaneval | 4.42 | 6.43 | 383 | 3.44x | 1.42x |
| math500 | 4.91 | 7.77 | 477 | 4.28x | 1.54x |
| mbpp | 5.23 | 8.04 | 478 | 4.29x | 1.46x |
| mtbench | 3.71 | 3.56 | 220 | 1.99x | 0.93x |
| ceval | 3.65 | 2.98 | 177 | 1.62x | 0.93x |
Built-in MTP baseline uses the official GLM-5.2 recipe (EAGLE, steps 5 / topk 1 /
draft tokens 6). This drafter is code/math-optimized: it delivers 1.4-1.5x over
the (already strong) built-in MTP on code and math workloads, while chat and
Chinese-language workloads slightly favor built-in MTP (see Limitations).
ceval (Chinese) is the weakest domain — the training corpus is English-dominant
(see Limitations).
<!-- Per-position acceptance table: not available from current bench output;
to be added in v1.1. -->
Also mirrored on ModelScope:
[UCloud-AILab/GLM-5.2-FP8-DFlash](https://modelscope.cn/models/UCloud-AILab/GLM-5.2-FP8-DFlash).
## Training Details
- Target model: GLM-5.2-FP8 (hidden 6144, 78 layers; drafter conditions on target
layers [1, 20, 38, 56, 75])
- Drafter: 5-layer block-diffusion transformer, block_size 16, **3.7B** total parameters
(**1.8B independently trained**; embed/lm_head reused from target, frozen, not trained,
included for standalone inference)
- Data: [JessieWei/GLM-5.2-FP8-nemotron-codealpaca](https://huggingface.co/datasets/JessieWei/GLM-5.2-FP8-nemotron-codealpaca)
— Nemotron-Post-Training-v2 + CodeAlpaca, ~800K samples (paper-specified recipe),
all responses regenerated by GLM-5.2-FP8 (non-thinking mode), max_length 3072
- Recipe: 6 epochs, AdamW with update clipping (StableAdamW-style), lr 6e-4 cosine (4% warmup), grad-clip 1.0, num_anchors 512,
loss_decay_gamma 7, pure cross-entropy (standard DFlash loss)
- Framework: SpecForge (offline hidden-state pipeline), FSDP2
## Limitations
- Trained on non-thinking-mode regenerated data; speedup under thinking-mode inference
has not been evaluated yet.
- Trained with `max_length` 3072: accept length degrades on very long prompts. For
long-context workloads, benchmark against the built-in MTP before choosing.
- Training corpus is English-dominant: acceptance length on Chinese-language
workloads is lower (ceval AL 2.98 vs 4.4-8.0 on English benchmarks).
## Acknowledgements
[DFlash](https://github.com/z-lab/dflash) (z-lab),
[SpecForge](https://github.com/sgl-project/SpecForge) /
[SGLang](https://github.com/sgl-project/sglang) (sgl-project).
GLM-5.2 by Zhipu AI.
## Citation
If you use this model, please cite:
```bibtex
@misc{ucloud2026glm52dflash,
title = {GLM-5.2-FP8-DFlash: A DFlash Speculative-Decoding Drafter for GLM-5.2-FP8},
author = {Wei, Xiaojun and {UCloud AILab}},
year = {2026},
howpublished = {\url{https://huggingface.co/UCloud-org/GLM-5.2-FP8-DFlash}}
}
```
This model is trained with the DFlash method — please also cite:
```bibtex
@misc{chen2026dflash,
title = {DFlash: Block Diffusion for Flash Speculative Decoding},
author = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
year = {2026}, eprint = {2602.06036}, archivePrefix = {arXiv},
primaryClass = {cs.CL}, url = {https://arxiv.org/abs/2602.06036}
}
```
|