--- base_model: Dream-org/Dream-v0-Instruct-7B library_name: peft tags: - base_model:adapter:Dream-org/Dream-v0-Instruct-7B - lora - transformers ---
# Dream-7B-Instruct-s1k-sft Dream-7B-Instruct-s1k-sft is a diffusion-based instruct model post-trained from [Dream-v0-Instruct-7B](https://huggingface.co/Dream-org/Dream-v0-Instruct-7B) on **[simplescaling/s1K](https://huggingface.co/datasets/simplescaling/s1K)**, using [MDLM](https://arxiv.org/abs/2406.07524) (masked diffusion) and trained with the [dLLM](https://github.com/ZHZisZZ/dllm) framework. ## Model Overview Dream-7B-Instruct-s1k-sft has the following features: - **Method:** [Masked Diffusion Language Modeling (MDLM)](https://arxiv.org/abs/2406.07524); - **Framework:** [dLLM](https://github.com/ZHZisZZ/dllm) - **Base model:** [Dream-v0-Instruct-7B](https://huggingface.co/Dream-org/Dream-v0-Instruct-7B) - **Dataset (SFT):** [simplescaling/s1K](https://huggingface.co/datasets/simplescaling/s1K) For broader training and ablation reporting in the dLLM ecosystem, see the [dLLM paper](https://arxiv.org/pdf/2602.22661). Eval notes: Metrics use confidence-threshold decoding (`alg: confidence_threshold`). The primary table is at `confidence_threshold` = 0.9; full grids sweep `confidence_threshold` ∈ {0.6, 0.7, 0.8, 0.9} with `max_new_tokens` ∈ {256, 512}. --- ## Primary results
Benchmark max_new_tokens=256
(Acc % | TPS)		 max_new_tokens=512
(Acc % | TPS)
GSM8K 81.80 | 2.30 84.31 | 2.56
HumanEval 54.27 | 2.53 53.05 | 3.24
MBPP 57.80 | 2.21 57.80 | 2.37
MATH 45.16 | 2.32 49.70 | 2.57
--- ## Threshold sweep
Benchmark τ=0.6
Acc | TPS		 τ=0.7
Acc | TPS		 τ=0.8
Acc | TPS		 τ=0.9
Acc | TPS
GSM8K 65.66 | 4.14 74.68 | 3.38 79.83 | 2.66 81.80 | 2.30
HumanEval 34.15 | 4.01 43.90 | 3.53 51.83 | 3.00 54.27 | 2.53
MBPP 41.80 | 3.53 49.40 | 2.71 55.80 | 2.36 57.80 | 2.21
MATH 37.02 | 3.64 41.80 | 3.12 44.48 | 2.67 45.16 | 2.32
max_new_tokens=256, columns sweep confidence_threshold ∈ {0.6, 0.7, 0.8, 0.9}
Benchmark τ=0.6
Acc | TPS		 τ=0.7
Acc | TPS		 τ=0.8
Acc | TPS		 τ=0.9
Acc | TPS
GSM8K 67.55 | 6.45 75.44 | 5.34 81.65 | 3.54 84.31 | 2.56
HumanEval 31.10 | 4.42 45.73 | 3.98 50.00 | 3.54 53.05 | 3.24
MBPP 42.20 | 5.60 50.20 | 3.20 56.80 | 2.59 57.80 | 2.37
MATH 38.72 | 4.68 44.10 | 3.90 48.22 | 3.13 49.70 | 2.57
max_new_tokens=512, columns sweep confidence_threshold ∈ {0.6, 0.7, 0.8, 0.9}