README.md

---
license: mit
library_name: transformers
pipeline_tag: text-generation
base_model:
  - zai-org/GLM-5.2-FP8
datasets:
  - JessieWei/GLM-5.2-FP8-nemotron-codealpaca
tags:
  - dflash
  - speculative-decoding
  - draft-model
  - block-diffusion
  - glm
---

# GLM-5.2-FP8-DFlash

[Paper](https://arxiv.org/abs/2602.06036) | [DFlash GitHub](https://github.com/z-lab/dflash) | [SpecForge](https://github.com/sgl-project/SpecForge)

DFlash block-diffusion speculative-decoding drafter for
[GLM-5.2-FP8](https://huggingface.co/zai-org/GLM-5.2-FP8) (743B MoE, 39B active).
Standard DFlash method (no extensions), trained with
[SpecForge](https://github.com/sgl-project/SpecForge) on the paper-specified data
recipe: ~800K samples of Nemotron-Post-Training-v2 + CodeAlpaca (code / math / chat),
all responses regenerated by GLM-5.2-FP8, 6 epochs — directly comparable to the DFlash
paper and the z-lab drafter series.

## Quick Start (SGLang)

```bash
python -m sglang.launch_server \
    --model-path zai-org/GLM-5.2-FP8 \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path UCloud-org/GLM-5.2-FP8-DFlash \
    --speculative-num-draft-tokens 16 \
    --tp-size 8 \
    --trust-remote-code
```

vLLM v0.20.1+ has native DFlash support and reads this checkpoint directly,
no conversion needed. Not yet runtime-verified on our infrastructure; the
results below were produced via SGLang.

## Evaluation

<!-- Sampling protocol (t=1.0/top_p 0.95) and per-position acceptance table
     to be added in v1.1. -->

### Mean accepted length & end-to-end speedup

Measured on live SGLang serving (concurrency 1, greedy decoding unless noted).

| Benchmark | AL (built-in MTP) | AL (DFlash) | DFlash throughput (tok/s) | Speedup vs vanilla | vs built-in MTP |
|-----------|-------------------|-------------|---------------------------|--------------------|-----------------|
| gsm8k     | 4.01 | 4.44 | 236 | 2.22x | 1.51x |
| humaneval | 4.42 | 6.43 | 383 | 3.44x | 1.42x |
| math500   | 4.91 | 7.77 | 477 | 4.28x | 1.54x |
| mbpp      | 5.23 | 8.04 | 478 | 4.29x | 1.46x |
| mtbench   | 3.71 | 3.56 | 220 | 1.99x | 0.93x |
| ceval     | 3.65 | 2.98 | 177 | 1.62x | 0.93x |

Built-in MTP baseline uses the official GLM-5.2 recipe (EAGLE, steps 5 / topk 1 /
draft tokens 6). This drafter is code/math-optimized: it delivers 1.4-1.5x over
the (already strong) built-in MTP on code and math workloads, while chat and
Chinese-language workloads slightly favor built-in MTP (see Limitations).

ceval (Chinese) is the weakest domain — the training corpus is English-dominant
(see Limitations).

<!-- Per-position acceptance table: not available from current bench output;
     to be added in v1.1. -->

Also mirrored on ModelScope:
[UCloud-AILab/GLM-5.2-FP8-DFlash](https://modelscope.cn/models/UCloud-AILab/GLM-5.2-FP8-DFlash).

## Training Details

- Target model: GLM-5.2-FP8 (hidden 6144, 78 layers; drafter conditions on target
  layers [1, 20, 38, 56, 75])
- Drafter: 5-layer block-diffusion transformer, block_size 16, **3.7B** total parameters
  (**1.8B independently trained**; embed/lm_head reused from target, frozen, not trained,
  included for standalone inference)
- Data: [JessieWei/GLM-5.2-FP8-nemotron-codealpaca](https://huggingface.co/datasets/JessieWei/GLM-5.2-FP8-nemotron-codealpaca)
  — Nemotron-Post-Training-v2 + CodeAlpaca, ~800K samples (paper-specified recipe),
  all responses regenerated by GLM-5.2-FP8 (non-thinking mode), max_length 3072
- Recipe: 6 epochs, AdamW with update clipping (StableAdamW-style), lr 6e-4 cosine (4% warmup), grad-clip 1.0, num_anchors 512,
  loss_decay_gamma 7, pure cross-entropy (standard DFlash loss)
- Framework: SpecForge (offline hidden-state pipeline), FSDP2

## Limitations

- Trained on non-thinking-mode regenerated data; speedup under thinking-mode inference
  has not been evaluated yet.
- Trained with `max_length` 3072: accept length degrades on very long prompts. For
  long-context workloads, benchmark against the built-in MTP before choosing.
- Training corpus is English-dominant: acceptance length on Chinese-language
  workloads is lower (ceval AL 2.98 vs 4.4-8.0 on English benchmarks).

## Acknowledgements

[DFlash](https://github.com/z-lab/dflash) (z-lab),
[SpecForge](https://github.com/sgl-project/SpecForge) /
[SGLang](https://github.com/sgl-project/sglang) (sgl-project).
GLM-5.2 by Zhipu AI.

## Citation

If you use this model, please cite:

```bibtex
@misc{ucloud2026glm52dflash,
  title        = {GLM-5.2-FP8-DFlash: A DFlash Speculative-Decoding Drafter for GLM-5.2-FP8},
  author       = {Wei, Xiaojun and {UCloud AILab}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/UCloud-org/GLM-5.2-FP8-DFlash}}
}
```

This model is trained with the DFlash method — please also cite:

```bibtex
@misc{chen2026dflash,
  title         = {DFlash: Block Diffusion for Flash Speculative Decoding},
  author        = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  year          = {2026}, eprint = {2602.06036}, archivePrefix = {arXiv},
  primaryClass  = {cs.CL}, url = {https://arxiv.org/abs/2602.06036}
}
```