anoane/DeepSeek-V4-Flash-0731-vllm-moet-sm120-cache

DeepSeek-V4-Flash-0731 — prebuilt vLLM-Moet cache for 1× RTX PRO 6000 (96 GB, SM120)

Prebuilt 2-bit expert planes + FP4 recovery packs for deepseek-ai/DeepSeek-V4-Flash-0731 (revision 9e165c30e2704aec5d9d593cce3eebd58bbef1cb), so this 159B MoE serves on a single RTX PRO 6000 Blackwell (96 GB, sm_120) with vLLM-Moet — no per-boot re-quantization of the 167 GB checkpoint, and no ~51 GiB host-RAM staging transient (which OOM-kills hosts with less than ~64 GB of RAM).

Built against upstream vLLM-Moet main @ de4af53 — the loader-skip is upstream now, so no fork is needed. Everything here is single-GPU: the artifacts are rank-sharded (tp1-rank0, rank0of1) and will not work on a multi-GPU split.

What actually gets quantized

Only the routed expert FFN weights are compressed. Everything that decides where tokens go and how they attend keeps checkpoint precision:

component	checkpoint	served as
routed experts (256 × 43 layers)	FP4	2-bit planes, with an FP4 pool restoring native precision for the hottest ~5%
shared expert, attention/MLA projections	FP8 e4m3 block [128,128]	FP8 (untouched)
router / mlp.gate, embeddings, lm_head	FP8 / BF16	untouched
MTP / DSpark blocks	—	excluded entirely (is_w2_layer: "main-model routed experts only")

So the FP4 pool is not an upgrade — it is un-doing our own 2-bit compression. An expert in the pool is bit-identical to the original checkpoint.

Repo contents

Shipped as zstd-compressed tar chunks:

chunks	contents	extracted
model.tar.zst.part.*	the full official 0731 checkpoint (48 shards + tokenizer/config)	156 GiB
planescache.tar.zst.part.*	planescache/v2/<key>/tp1-rank0/ — 43 layers × 6 parts + meta	202 GiB
store.tar.zst.part.*	moet_store/ — base, delta, fp4 v2 packs + sidecars	339 GiB

Download & extract

REPO=anoane/DeepSeek-V4-Flash-0731-vllm-moet-sm120-cache
fetch() {
  for p in $(curl -sfL "https://huggingface.co/api/models/$REPO/tree/main?recursive=true" \
            | python3 -c 'import json,sys;[print(e["path"]) for e in json.load(sys.stdin) if e["type"]=="file"]' \
            | grep "^$1" | sort); do
    curl -sfL --retry 5 "https://huggingface.co/$REPO/resolve/main/$p"
  done
}
mkdir -p /data/ds4 && cd /data/ds4
fetch model       | zstd -d -T4 | tar -x --sparse     # -> DeepSeek-V4-Flash-0731/
fetch planescache | zstd -d -T4 | tar -x --sparse     # -> planescache/
fetch store       | zstd -d -T4 | tar -x --sparse     # -> moet_store/

⚠️ Mandatory: pin the checkpoint identity

The persistent-cache identity hashes a per-shard stat manifest (inode, size, mtime, ctime) — it can never match after files are copied or downloaded. Without the pin the cache silently MISSES, the engine re-stages the whole checkpoint (~51 GiB host RAM) and small hosts get OOM-killed with no moe_w2 log line to explain it. The sanctioned override (moe_w2_planes_cache._ckpt_id), value taken from this cache's own meta.json:

-e VLLM_MOE_W2_CKPT_ID=8863d864b7c75cde7493abc5899bdfe9e6dd16f6dab445961416f6e407c842b8

One env var covers the planescache and all packs. It is in every command below.

Build the engine

git clone https://github.com/kacper-daftcode/vLLM-Moet && cd vLLM-Moet && git checkout de4af53
DOCKER_BUILDKIT=1 docker build -f Dockerfile.sm120-v024 -t vllm-moet-sm120:v026 .

Turn ECC off (sudo nvidia-smi -e 0 + reboot). Server-Edition cards ship with it enabled; it costs ~6 GiB and the GPU-resident planes need every byte.

Serve — three profiles

One base command, three settings of the FP4 recovery-pool scheduler. Everything else is identical.

profile	scheduler setting	decode	quality (0–30)	quality vs #1
1 · max quality ⭐	DELTA_POLICY=freq, no damping	48.9 tok/s	12.27	—
2 · quality / speed	+ PROMO_HYST=8	75.0 tok/s (+53%)	10.90	−1.37 (≈ −11%)
3 · max speed	DELTA_POLICY=need	83.2 tok/s (+70%)	≈ half of #1	≈ −50%

Quality is a blind 0–30 rubric (correctness 12 / completeness 10 / engineering 8) over hard systems-programming tasks. Profiles 1 vs 2 are separated by 30 paired observations across 5 independent seeds; profile 3 by a 5-rung panel. Decode is single-stream at 256K context and is reproducible to ±1 tok/s across seeds.

Pick profile 1 unless you have a reason not to. Profile 2 is a real trade, not a free win: ~11% quality for +53% throughput. Profile 3 is a large quality loss for the last 11% of speed and is only sensible when output quality is genuinely not a criterion.

Profile 1 — max quality (RECOMMENDED, production)

docker run -d --name ds4-quality --gpus '"device=0"' --network host --ipc host --shm-size 32g \
  -v /data/ds4/DeepSeek-V4-Flash-0731:/model:ro \
  -v /data/ds4/planescache:/planescache \
  -v /data/ds4/moet_store:/store \
  -e VLLM_MOE_W2=1 \
  -e VLLM_MOE_W2_PLANES_CACHE=/planescache \
  -e VLLM_MOE_W2_STORE_DIR=/store \
  -e VLLM_MOE_W2_CKPT_ID=8863d864b7c75cde7493abc5899bdfe9e6dd16f6dab445961416f6e407c842b8 \
  -e VLLM_MOE_W2_DELTA_POLICY=freq \
  -e VLLM_MOE_W2_DELTA_GB=7 \
  -e VLLM_MOE_W2_GATE=0 \
  -e VLLM_MOE_W2_FORCE_POOL=1 \
  -e VLLM_MOE_W2_FORCE_RESIDENT=1 \
  -e VLLM_USE_BREAKABLE_CUDAGRAPH=0 \
  -e VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0 \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  vllm-moet-sm120:v026 \
  --model /model --served-model-name deepseek-v4-flash --trust-remote-code \
  --kv-cache-dtype fp8 --block-size 256 --max-model-len 262144 \
  --gpu-memory-utilization 0.99 --kv-cache-memory-bytes 2147483648 \
  --max-num-batched-tokens 1024 --max-num-seqs 1 \
  --tokenizer-mode deepseek_v4 --no-scheduler-reserve-full-isl \
  --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE","custom_ops":["all"]}' \
  --port 8000

Enabling the gate (VLLM_MOE_W2_GATE=1 GATE_TAU=0.15 GATE_FP_MAX=1) costs 13% throughput for no measured quality gain, and it is the arm that failed to terminate on one hard rung. Leave it off in every profile.

Profile 2 — best quality / speed ratio

Take the profile-1 command and add one line:

  -e VLLM_MOE_W2_PROMO_HYST=8 \

75.0 tok/s (+53.4%) for a measured −1.37/30 (≈ −11% relative).

Hysteresis damps the promotion scheduler so the pool stops thrashing slots. That recovers most of the throughput the FP4 pool costs, and it does have a quality price — the honest label is "~10% quality cost for +53% speed", not "equivalent".

The evidence, because this is the one profile where the trade needs to be explicit:

• Primary estimate −1.37, seed-blocked 90% CI [−2.48, −0.26] (n = 5 seeds); conservative rung-blocked CI [−2.93, +0.19]. P(true effect is a real loss) ≈ 0.97; P(loss > 3 points) ≈ 0.017.
• The deficit is broad-based, not one bad task: 4 of 5 seeds and 5 of 6 rung types negative, and the sign never flips under any leave-one-rung-out or leave-one-seed-out.
• 30% of cells were exact ties and 53% fell within 2 points — a typical request shows no visible difference. The failure category never changes; what changes is how much of a hard answer is broken.
• Equivalence within ±1.2 is not demonstrated (TOST p = 0.60), and a quality gain is ruled out (p = 0.0003). So: cheaper and slightly worse, definitively not free.

PROMO_HYST=4 is not a gentler setting — speed saturates at 4 (4/8/16/32 all ≈ 76 tok/s) while measuring worse than 8 (−2.72, 90% CI [−4.77, −0.67]). If you want this profile, use 8.

Profile 3 — max speed

Take the profile-1 command and change one line:

  -e VLLM_MOE_W2_DELTA_POLICY=need \      # replaces DELTA_POLICY=freq

83.2 tok/s (+70%) for roughly half the quality.

need fills the FP4 pool on demand only, with no background promotion, so the hottest experts are never pre-staged. Measured −7.20/30 against profile 1 (90% CI [−11.06, −3.34]), losing on 5 of 5 rungs. The failures are structural rather than cosmetic — answers arriving with no type environment at all, no leader role in a Raft implementation, terminating before the first line of allocator code.

Note the marginal trade: profile 3 buys only +11% speed over profile 2 and pays about 4× more quality for it. If your max-speed profile has any quality bar at all, use profile 2 and take 75 tok/s. Profile 3 is documented because it is the true endpoint of the frontier, not because it is usually the right choice.

Per-request sampling (all profiles): temperature=1.0, top_p=1.0, min_p=0.05. Never temperature=0 on a reasoning model — it loops.

Why each knob (max-quality lens)

• VLLM_MOE_W2=1 — the 2-bit routed-expert path. The native checkpoint (167 GB) does not fit 96 GB.
• VLLM_MOE_W2_PLANES_CACHE=/planescache — boot the 2-bit planes straight from this repo (loader-skip): no re-quant, no host staging, no swap. Quality-neutral by construction.
• VLLM_MOE_W2_CKPT_ID=… — identity pin, mandatory for any transferred cache (above).
• VLLM_MOE_W2_STORE_DIR=/store — the FP4 pack backs the recovery pool via pread, so the pool does not need 142 GiB of pinned host RAM.
• VLLM_MOE_W2_DELTA_POLICY=freq ⭐ — the quality knob. Keeps the hottest experts resident at native FP4. Measured: ~57% of all token→expert routings served at native precision. This is what prevents the long-generation decoherence documented below.
• VLLM_MOE_W2_DELTA_GB=7 — pool size. 512 slots (6 GiB) already reach the ~57% coverage ceiling; 768 slots (9 GiB) measured no better (routing has a long tail), and 10 GiB does not fit at 256K. 7 GiB is free relative to 6.
• VLLM_MOE_W2_GATE=0 ⭐ — the confidence gate re-runs low-confidence steps with routed experts promoted to FP4. Measured across three studies it buys nothing here. Within a freq pool the gate-on/gate-off difference is +0.05/30 (p=0.95) once a runaway rung is excluded, while costing 13% decode — and the gate-on arm is the one that failed to terminate (>55k tokens on a type-inference rung). Its defaults are worse still: default TAU fires on ~every token and FP_MAX=3 allows three extra full forwards, together 2.4× decode for no gain. Broad pool coverage, not gate firing, is what protects quality.
• VLLM_USE_BREAKABLE_CUDAGRAPH=0 — vLLM auto-enables this for DeepseekV4ForCausalLM, which disables the torch.compile pipeline (CompilationMode.NONE). Setting it to 0 restores VLLM_COMPILE (+5%).
• PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True — removes ~0.5 GiB of allocator fragmentation; without it the 256K config CUDA-OOMs at load.
• VLLM_MOE_W2_FORCE_RESIDENT=1 / VLLM_MOE_W2_FORCE_POOL=1 — consent valves for the boot guards, whose reserve estimates are deliberately conservative and refuse knife-edge 1×96 GB configs that do in fact serve. A real shortfall still fails safely at load.
• VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0 — the graph profiler over-reserves ~2 GiB vs a real ~0.09 GiB graph pool; reclaiming it funds the FP4 pool.
• --kv-cache-memory-bytes 2147483648 — pin the KV pool. 256K needs 1.44 GiB; util-based sizing overcommits on this base and OOMs at load.
• --kv-cache-dtype fp8 --block-size 256 — DS4's fp8_ds_mla attention requires fp8 KV (~1.8 KB/token, so a 256K window costs under 0.5 GiB — context here is cheap, precision isn't).
• --max-model-len 262144 — 256K. Validated: all three needle depths pass at 256K and none pass at 288K — 256K is a hard ceiling on this checkpoint.
• --max-num-seqs 1 — single stream. Concurrency multiplies the gate's fire floor and the pool's working set.
• CUDA graphs: keep them. --enforce-eager costs 2.9× (27.0 → 9.4 tok/s) and saves only 0.09 GiB. Never use it here.
• speculative decoding: off. 0731 replaced the old single MTP head with 3 DSpark blocks; the old {"method":"deepseek_mtp"} config cannot load them.
• VLLM_MOE_W2_PROMO_HYST — leave unset for max quality; =8 is profile 2. Damping the pool's promotion scheduler stops slot thrashing and is worth +53% decode, but it lowers steady-state FP4 coverage, and coverage is the entire quality mechanism. Measured cost −1.37/30 over 30 paired observations. Keep it out of the max-quality profile: the direction is settled (P ≈ 0.97 that it is a real loss) even though the magnitude is not, and the max-quality slot's rule is that any doubt favours the incumbent. =4 is strictly worse than =8 for the same speed.
• VLLM_MOE_W2_DELTA_PROMOTE=2 — a middle damping setting. Measured 63.5 tok/s at −2.11/30: slower and worse than PROMO_HYST=8. Dominated; no reason to use it.
• Speculative decoding is impossible with VLLM_MOE_W2. DSpark (k=1/2/3) fails at engine init: ValueError: VLLM_MOE_W2 is not integrated with Model Runner V2 (mandatory base replay and gate hooks). The 2-bit expert path and Model Runner V2 are mutually exclusive, so the 0731 checkpoint's DSpark blocks cannot be used here regardless of the dspark_* config keys. N-gram spec decode does boot, if you want to experiment.
• --linear-backend / --moe-backend: leave on auto. flashinfer_b12x, cutlass and moe flashinfer_b12x were all tested and fail at engine init on this sm120 + fp8-block stack.

Measured (this cache, this engine, 1× RTX PRO 6000, 256K ctx, single stream)

configuration	decode tok/s	note
profile 1 — max quality	48.9	the command above; ±0.2 across 5 seeds
profile 2 — quality/speed (PROMO_HYST=8)	75.0	+53.4%; −1.37/30 quality
profile 3 — max speed (DELTA_POLICY=need)	83.2	+70%; −7.20/30 quality
PROMO_HYST=4 / 16 / 32	~76	speed saturates at 4; quality worse than 8
DELTA_PROMOTE=2	63.5	dominated — slower and worse than PROMO_HYST=8
9 GiB pool (DELTA_GB=9)	48.4	+0.44/30 (p=0.87) — inside the noise floor, not worth 2 GiB
coverage only + gate on	42.2	gate costs 13% for no measured gain
gate at default TAU, FP_MAX=3	33.0	fires ~every token — the old README's shape
default gate + DELTA_POLICY left at freq default*	~25	the 3× trap
no FP4 recovery at all (GATE=0, no coverage)	85.4	fast and broken — see quality study
--enforce-eager (no CUDA graphs)	9.4	never do this

* VLLM_MOE_W2_DELTA_POLICY silently defaults to freq only when no host base-cache is configured; with freq and the gate on, the background promoter churns the pool at +1536/−1536 slots per 320 ms ≈ 57 GB/s of host→device traffic, saturating PCIe gen5 ×16. Verify with VLLM_MOE_W2_DELTA_TRACE=1: the [delta] tick … window +N/−N line should be small. (This recipe keeps freq deliberately — with the gate rarely firing, the coverage it provides is the quality mechanism; it is the combination with an always-firing gate that is toxic.)

VRAM at 256K: ~93.8 GiB of 96 — 72.6 GiB of 2-bit planes + 7 GiB FP4 pool + KV + activations.

Context (needle-in-a-haystack, unique code at depths 0.05 / 0.50 / 0.95)

length	result
64K	3/3 depths
256K	3/3 depths ⭐ recommended --max-model-len 262144
288K	0/3 depths — hard cliff, do not serve above 256K

Quality study — where the FP4 budget should go

Six serving recipes generated answers to a 6-rung extremely hard coding ladder (lock-free MPMC queue with per-atomic memory-order justification and a linearizability proof; incremental Earley parser with SPPF sharing and an invalidation proof; SSA register allocator with Briggs+George coalescing, aliasing and remat; Raft with joint consensus and the leader-removed-by-its-own-config-change case; AVX-512 SpMV with bit-identical cross-thread reduction and a forward error bound; HM + row polymorphism + fundeps + rank-2 inference). Same seed everywhere. Answers were anonymized, shuffled, and graded blind by Claude Opus 5, two independent judges per rung.

Six recipes × six rungs, blind-graded (0–30) by Claude Opus 5:

recipe	quality /30	tok/s
broad FP4 coverage (freq pool), gate off	11.00	49.1
gate TAU=0.5, FP_MAX=1	10.29	38.5
gate default, FP_MAX=3	9.88	33.0
gate TAU=0.3, FP_MAX=1	8.79	56.2
gate TAU=0.15, FP_MAX=1	8.08	71.5
no FP4 recovery at all	4.96	85.4

Result 1 — FP4 recovery is mandatory (this one is solid). Serving with no recovery scored worst on 6 of 6 rungs (+4.65/30 for any recovery, paired t(5)=3.53, p=0.017). It does not degrade gracefully, it decoheres: an 80-line Raft function containing only comments; an AVX-512 kernel body that says "I'll now write the final code" and then closes the brace, twice. The 85.4 tok/s config is unusable.

Result 2 — a rare gate on top of coverage: directionally positive, not proven. A dedicated head-to-head (same seed, untruncated answers, 2 judges/rung, pairing self-verified) gave the gate +1.92/30 (13.42 vs 11.50), winning 4 of 6 rungs, t(5)=2.203, p=0.079, exact permutation p=0.125, dz=0.90. Not significant at n=6. It is not a verbosity artifact — correlation between length delta and score delta is −0.55 (the gate's largest win came while writing 1,256 characters fewer). We enable it because the cost is certain and bounded (13.5%, ~+64 s on a 20k-token answer) while the benefit is uncertain but positively signed, and this is a single-stream quality box. Turn it off the moment throughput matters.

Calibration, honestly stated: the grand mean is 12.5/30 — both arms fail most of these rubrics. The gate converts a worse failure into a less-bad one (1 of 24 registers coloured vs an infinite loop; buggy threading vs absent threading), not a failure into a success. Settling the ordering properly would need ~12–18 distinct rungs, not 6.

Methodology note: an earlier run of this comparison reached the opposite conclusion and was invalid — answers were clipped at 22k chars, which is length-biased against the more verbose arm (one answer's __main__ block sat at char 25,181 and was graded "missing"). Re-run untruncated, the sign flipped. If you reproduce this, do not clip judge inputs.

The one statistically solid result: FP4 recovery is not optional. Disabling it entirely (gate off and no pool coverage) scored worst on 6 of 6 rungs (+4.65/30 for any recovery, paired t(5)=3.53, p=0.017). It does not degrade gracefully — it decoheres: judges found it emitting plans instead of code (an 80-line Raft function of pure comments; an AVX-512 kernel body that says "I'll now write the final code" and then closes the brace, twice).

Beyond that, be skeptical of the ordering. Within the top configs, per-task luck (sd 2.95) exceeds the config signal (sd 1.98); F(5,25)=2.82, p=0.037 against the correct error term. A decisive ranking would need ~25–30 rungs × ~5 generations per cell (750–900 generations), not 36. The recommendation above rests on avoiding catastrophic failure modes, not on a 2-point lead.

Regenerating from scratch

One Config-A-shaped boot on a big-RAM box (≥128 GiB) with VLLM_MOE_W2_PLANES_CACHE and VLLM_MOE_W2_STORE_DIR pointed at empty writable dirs writes planescache/ + the delta/fp4 packs (~13 min on 2× RTX PRO 6000 / 288 GB); a second boot with VLLM_MOE_W2_BASE_CACHE_GB=14 adds the base pack. Generate with TP1 on one GPU — the artifacts are rank-sharded.

Pins

what	value
model	deepseek-ai/DeepSeek-V4-Flash-0731 @ 9e165c30e2704aec5d9d593cce3eebd58bbef1cb
vLLM-Moet	kacper-daftcode/vLLM-Moet @ de4af53
vllm fork (patch SOURCE)	db13b2e9cd942cf015f3137c8b51644bbe67fb06
quantizer ABI	tsym4-e8m0-fragmajor-a32-v2
ckpt_id (pin this)	8863d864b7c75cde7493abc5899bdfe9e6dd16f6dab445961416f6e407c842b8

Credit: the 2-bit/FP4/SM120 engine, the SM120 SASS toolchain (blackwell-isa, cubit) and the entire moet design are @kacper-daftcode's work. This repo packages the DS4-0731 cache and the measured single-GPU serving recipe.Two blind studies, Claude Opus 5 as judge, answers shuffled and anonymised, nothing truncated.

Study 1 — is FP4 recovery needed at all? (6 recipes × 6 rungs)

recipe	quality /30	tok/s
broad FP4 coverage (freq pool), gate off	11.00	49.1
gate TAU=0.5, FP_MAX=1	10.29	38.5
gate default, FP_MAX=3	9.88	33.0
gate TAU=0.3, FP_MAX=1	8.79	56.2
gate TAU=0.15, FP_MAX=1	8.08	71.5
no FP4 recovery at all	4.96	85.4

The solid result: FP4 recovery is mandatory. No-recovery lost 6 of 6 rungs (+4.65/30 for any recovery, paired t(5)=3.53, p=0.017). It does not degrade gracefully — it decoheres: an 80-line Raft function of pure comments; an AVX-512 kernel body that says "I'll now write the final code" and closes the brace, twice. That 85.4 tok/s configuration is unusable.

Study 2 — the confidence gate: three studies, no benefit

study	design	gate effect	verdict
6 rungs, token-capped (invalid)	6 recipes	+1.92/30, p=0.079	superseded — the cap truncated 1–2 rungs per arm
12 fresh rungs, uncapped	pre-registered confirmation	+1.40/30, p=0.168, dz=0.43	not resolved; effect below the design's resolution floor
6 rungs, uncapped, matched pool	freq 7 GiB, gate on vs off	+0.05/30, p=0.95 (excl. runaway)	flat null

Conclusion: leave the gate off. It costs 13% throughput, shows no measurable quality gain on matched configurations, and carries tail risk — the gate-on arm is the one that failed to terminate (>55,000 tokens on type inference, killed at 21 min).

The one genuinely strong statistical signal is an interaction: rung × gate F(11,23)=9.43, p=3.9e-6. The gate helps some task types (concurrent-mark GC +7.0, LSM compaction +6.0, CUDA flash-attention +4.5) and hurts others (SIMD UTF-8 −4.0, wait-free snapshot −2.25). If your workload is dominated by algorithm-and-proof work you may still want it; for general use the throughput is worth more than a coin-flip.

Study 3 — pool policy and size (uncapped, 6 configs × 6 rungs)

config	quality /30	tok/s
freq 7 GiB, gate off ⭐	9.58	48.9
freq 7 GiB, gate on	8.46	42.5
need, default gate	7.83	43.6
freq 6 GiB, gate off	7.46	49.7
need, TAU=0.3	5.67	63.3
no FP4 recovery	2.96	88.2

• freq beats need by +1.75/30 on 5 of 6 rungs (p=0.149 — directional, not significant). Note the trap: need + TAU 0.3 is the fastest configuration measured (63.3 tok/s) but the worst of the working configs. Speed there is bought with quality, not for free.
• Pool size 7 GiB > 6 GiB by +2.12 (directional). Coverage saturates ~57% of routings, so 9–10 GiB buys nothing.
• No-recovery replicates as catastrophic: any recovery beats none by +4.84/30, losing 6/6 rungs, permutation p=0.0018. This matches the capped study's +4.65 almost exactly and is the one result that is solid across every design.

Study 4 — the scheduler-damping trade, across 5 independent seeds

The profile-2 question ("does PROMO_HYST=8 cost quality, and how much?") could not be settled by single-seed data, and the reason is worth recording: generation at a fixed seed is deterministic, so re-judging the same outputs with more judges re-measures one draw. A variance decomposition put ~90% of the paired-delta variance in generation and ~10% in judging — meaning extra judges shrink the standard error by under 2%, and only extra seeds add real information.

So the contrast was re-run at seeds 42–46, 6 rungs each = 30 paired observations, one judge per cell on a rotating lens (lens severity cancels inside a pair, so this spends the budget on draws).

analysis	n	mean Δ	90% CI	p
pooled (overstates precision — df, not SE)	30	−1.37	[−2.49, −0.24]	0.047
seed-blocked (primary)	5	−1.37	[−2.48, −0.26]	0.058
rung-blocked (conservative, generalizing)	6	−1.37	[−2.93, +0.19]	0.138

Seed variance estimated at zero (F(4,20)=0.63, p=0.65) — the seeds are interchangeable and no single draw carries the result. Leave-one-seed-out spans only −1.08 to −1.75.

What single-seed data got wrong. On seed 42 the entire contrast appeared to hinge on the lock-free-queue rung: dropping it flipped the estimate from −1.93 to +0.25, which looked like evidence that hysteresis was free. With five independent draws of that rung, that flip does not replicate — excluding the rung entirely still leaves −0.88, and the per-seed excl-rung deltas are −1.40, −0.20, 0.00, −1.20, −1.60, none positive. The earlier "it's all one rung" reading was itself a single-draw artifact. This is the clearest demonstration in the campaign of why the seed axis matters and the judge axis does not.

Still not established: whether the true cost is inside or outside the ±1.2 measurement floor (P ≈ 0.62 / 0.38 — a coin flip; ~10 seeds would settle it), and whether the deficit generalizes beyond these six task types (the rung-blocked interval includes zero).

Definitive cross-stack panel (one panel, same 3 judges, identical rungs)

Earlier comparisons ran as separate workflows with slight calibration drift. This one scores every stack together, 3 blind judges per rung, nothing truncated.

stack	quality /30	tok/s	wall-clock, 6 rungs	artifacts that run
DS4-0731, 2-bit + 7 GiB FP4 recovery ⭐	11.86	~49	11.5 min	3/6
Qwen3.6-27B BF16 (unquantized reference)	11.14	~28	46.5 min	2/6
Laguna-S-2.1 NVFP4	4.90	148–190	1.6 min	2/6
DS4-0731, no FP4 recovery	4.32	~88	4.4 min	1/6

17 of 17 judgments place {DS4-with-recovery, Qwen} above {laguna, DS4-without} — p = 5.9e-14. Judge reliability ICC(2,1) = 0.941, so rung count is the binding constraint, not judging.

1. The FP4 recovery tier is the entire advantage. With vs without: +7.54/30, 6/6 rungs, p = 0.0009 (Holm 0.0027), dz = 2.87. It costs ~427 s across the suite and buys 7.5 points — 57 s per quality point, the best marginal rate of any knob measured in this campaign.

2. Without it, this stack is pointless. No-recovery vs laguna: −0.58/30, p = 0.622, statistically equivalent within ±3/30 — while being 2.7× slower. If the pool fails to load you are running a strictly worse production model. Check the [delta] tick line is present.

3. 2-bit + recovery ≈ unquantized BF16. vs Qwen3.6-27B BF16: +0.72/30, p = 0.658; equivalence established within ±4/30. A 159B MoE at 2 bits matches a 27B dense at 16 bits here. (Model+quant together, not a quantization isolate — this says nothing about DS4-2-bit vs DS4-full-precision.) The null hides real structure: DS4 wins the systems/numerics rungs (lock-free, regalloc, SIMD), Qwen wins the distributed/formal ones (parser, Raft, types).

4. Throughput still favours production. Normalized quality-per-wall-clock-second: laguna 1.00, DS4-with-recovery 0.34, Qwen 0.09. Use laguna as the default and escalate hard problems to this stack — you pay ~7× wall-clock for ~2.4× quality.

**Calibration, stated plainly:**Calibration, stated plainly: grand means are 9.6–12.6 out of 30. across 24 artifacts in the definitive panel, exactly one was verified to run and produce correct output (this stack's lock-free MPMC queue: 200k ops, 4 producers / 4 consumers, ThreadSanitizer-clean). Judges noted that "compiles/runs" mostly means "does not throw", not "works". This ranks failure modes.

Regenerating from scratch

One Config-A-shaped boot on a big-RAM box (≥128 GiB) with VLLM_MOE_W2_PLANES_CACHE and VLLM_MOE_W2_STORE_DIR pointed at empty writable dirs writes planescache/ + the delta/fp4 packs (~13 min on 2× RTX PRO 6000 / 288 GB); a second boot with VLLM_MOE_W2_BASE_CACHE_GB=14 adds the base pack. Generate with TP1 on one GPU — the artifacts are rank-sharded.

Pins

what	value
model	deepseek-ai/DeepSeek-V4-Flash-0731 @ 9e165c30e2704aec5d9d593cce3eebd58bbef1cb
vLLM-Moet	kacper-daftcode/vLLM-Moet @ de4af53
vllm fork (patch SOURCE)	db13b2e9cd942cf015f3137c8b51644bbe67fb06
quantizer ABI	tsym4-e8m0-fragmajor-a32-v2
ckpt_id (pin this)	8863d864b7c75cde7493abc5899bdfe9e6dd16f6dab445961416f6e407c842b8

Credit: the 2-bit/FP4/SM120 engine, the SM120 SASS toolchain (blackwell-isa, cubit) and the entire moet design are @kacper-daftcode's work. This repo packages the DS4-0731 cache and the measured single-GPU serving recipe.