JessieWei commited on
Commit
2261b3c
·
verified ·
1 Parent(s): 3b71a39

docs: self-citation, update-clipping recipe note, long-context limitation

Browse files
Files changed (1) hide show
  1. README.md +16 -1
README.md CHANGED
@@ -84,7 +84,7 @@ Also mirrored on ModelScope:
84
  - Data: [JessieWei/GLM-5.2-FP8-nemotron-codealpaca](https://huggingface.co/datasets/JessieWei/GLM-5.2-FP8-nemotron-codealpaca)
85
  — Nemotron-Post-Training-v2 + CodeAlpaca, ~800K samples (paper-specified recipe),
86
  all responses regenerated by GLM-5.2-FP8 (non-thinking mode), max_length 3072
87
- - Recipe: 6 epochs, AdamW, lr 6e-4 cosine (4% warmup), grad-clip 1.0, num_anchors 512,
88
  loss_decay_gamma 7, pure cross-entropy (standard DFlash loss)
89
  - Framework: SpecForge (offline hidden-state pipeline), FSDP2
90
 
@@ -92,6 +92,8 @@ Also mirrored on ModelScope:
92
 
93
  - Trained on non-thinking-mode regenerated data; speedup under thinking-mode inference
94
  has not been evaluated yet.
 
 
95
  - Training corpus is English-dominant: acceptance length on Chinese-language
96
  workloads is lower (ceval AL 2.98 vs 4.4-8.0 on English benchmarks).
97
 
@@ -104,6 +106,19 @@ GLM-5.2 by Zhipu AI.
104
 
105
  ## Citation
106
 
 
 
 
 
 
 
 
 
 
 
 
 
 
107
  ```bibtex
108
  @misc{chen2026dflash,
109
  title = {DFlash: Block Diffusion for Flash Speculative Decoding},
 
84
  - Data: [JessieWei/GLM-5.2-FP8-nemotron-codealpaca](https://huggingface.co/datasets/JessieWei/GLM-5.2-FP8-nemotron-codealpaca)
85
  — Nemotron-Post-Training-v2 + CodeAlpaca, ~800K samples (paper-specified recipe),
86
  all responses regenerated by GLM-5.2-FP8 (non-thinking mode), max_length 3072
87
+ - Recipe: 6 epochs, AdamW with update clipping (StableAdamW-style), lr 6e-4 cosine (4% warmup), grad-clip 1.0, num_anchors 512,
88
  loss_decay_gamma 7, pure cross-entropy (standard DFlash loss)
89
  - Framework: SpecForge (offline hidden-state pipeline), FSDP2
90
 
 
92
 
93
  - Trained on non-thinking-mode regenerated data; speedup under thinking-mode inference
94
  has not been evaluated yet.
95
+ - Trained with `max_length` 3072: accept length degrades on very long prompts. For
96
+ long-context workloads, benchmark against the built-in MTP before choosing.
97
  - Training corpus is English-dominant: acceptance length on Chinese-language
98
  workloads is lower (ceval AL 2.98 vs 4.4-8.0 on English benchmarks).
99
 
 
106
 
107
  ## Citation
108
 
109
+ If you use this model, please cite:
110
+
111
+ ```bibtex
112
+ @misc{ucloud2026glm52dflash,
113
+ title = {GLM-5.2-FP8-DFlash: A DFlash Speculative-Decoding Drafter for GLM-5.2-FP8},
114
+ author = {Wei, Xiaojun and {UCloud AILab}},
115
+ year = {2026},
116
+ howpublished = {\url{https://huggingface.co/UCloud-org/GLM-5.2-FP8-DFlash}}
117
+ }
118
+ ```
119
+
120
+ This model is trained with the DFlash method — please also cite:
121
+
122
  ```bibtex
123
  @misc{chen2026dflash,
124
  title = {DFlash: Block Diffusion for Flash Speculative Decoding},