olka-fi/Laguna-S-2.1-MXFP4-dspark

Laguna-S-2.1-MXFP4 — DSpark speculator head

A DSpark speculative-decoding draft head for olka-fi/Laguna-S-2.1-MXFP4 (118B-A8B MoE verifier, MXFP4). Trained with vllm-project/speculators.

Usage (vLLM)

Exact command used to serve this head and produce the numbers above (B300, single GPU):

vllm serve olka-fi/Laguna-S-2.1-MXFP4 \
  --speculative-config '{"model":"olka-fi/Laguna-S-2.1-MXFP4-dspark","method":"dspark","num_speculative_tokens":4}' \
  --port 8000 --gpu-memory-utilization 0.9 

DSpark extends DFlash's anchored-block drafting with a Markov head (conditions each draft position on the previously drafted token within the block) and a confidence head (predicts per-position acceptance probability).

Evaluation

Metric	Result	Notes
GSM8K (full test, 1319, temp 0)	95.5%	chat-template CoT harness, num_speculative_tokens=4; matches base verifier (~94.5%) — spec decode is lossless
Decode speedup	~2×	vs. no-draft baseline (cudagraphs, B300)

Head architecture

Algorithm	dspark (anchored block, Markov + confidence heads)
Draft decoder	3 × Qwen3-style layers, hidden 3072, sliding-window 512 on all draft layers
Block size	8
Aux hidden layers (from verifier)	[2, 24, 45] + final
Draft vocab	32000 (EAGLE-3 vocab compression)
Markov rank	256 (vanilla)
Params	~941M (bf16), ~1.88 GB

Training

• Data: mlabonne/open-perfectblend, 22k samples, seq-len 16000.
• Regime: offline hidden-state distillation (teacher-forced / off-policy), 10 epochs.
• Loss: {"ce": 0.1, "tv": 0.9}; Muon + AdamW.

Requirements / caveats

• Needs a vLLM build with dspark support
• On NVIDIA Blackwell B300 (sm100) VLLM_FORCE_MXFP4_MARLIN=1 is required — the CUTLASS MXFP4 MoE kernel is fine for plain generation but faults under DSpark's draft-block shapes; the Marlin MXFP4 backend is stable.