Update DGX Spark 200K serving recipe

README.md

@@ -1,215 +1,92 @@
 ---
 license: mit
 library_name: transformers
-base_model: deepseek-ai/DeepSeek-V4-Flash
+pipeline_tag: text-generation
 tags:
 - deepseek-v4
 - mixture-of-experts
 - reap
+- dgx-spark
+- vllm
+- long-context
+- fp8
+- mxfp4
 - experimental
-- text-generation
-private: true
+base_model: deepseek-ai/DeepSeek-V4-Flash
 ---
 
-# DeepSeek-V4-Flash-162B-codex-K144-REAP
-
-**Experimental checkpoint, not ready for production use. Keep private unless explicitly approved for release.**
-
-This is a Routing-Enhanced Activation Pruning (REAP) derivative of
-[`deepseek-ai/DeepSeek-V4-Flash`](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash).
-It keeps **144 / 256 routed experts per routed layer** in a compact MoE layout.
-
-The goal of this checkpoint is to fit a higher-quality DeepSeek-V4-Flash REAP in a tighter serving envelope than K160 while preserving long-context, tool/JSON, and formatting behavior.
-
-## Status
+# Deepseek-V4-Flash-162B-REAP
 
-- Repo visibility: private
-- Readiness: experimental validation checkpoint
-- Main known issue: long visual-format generations can still trigger conservative repetition detectors, mostly on diagram separator patterns
-- No observed `U+FFFD` replacement-character corruption in the latest targeted probe
-- Recommended decoding for the targeted formatting probe: `temperature=1.0`, `top_p=1.0`, `repetition_penalty=1.03`
+This is the 162B / K144 REAP-pruned DeepSeek V4 Flash model. The validated single-DGX Spark serving recipe is maintained here:
 
-## Build Summary
+- GitHub: https://github.com/0xSero/deepseek-v4-flash-spark-200k
+- Docker image: `ghcr.io/0xsero/deepseek-v4-flash-spark-vllm:cutlass451-g27`
+- Model repo used by the recipe: `0xSero/DeepSeek-V4-Flash-162B-codex-K144-REAP`
+- Validated revision: `d663e8fb16809f6619000648b187b257249ed824`
 
-| Field | Value |
-|---|---:|
-| Base model | `deepseek-ai/DeepSeek-V4-Flash` |
-| Target label | `162B-codex-K144` |
-| Estimated total parameters | `162.801B` |
-| Original routed experts / layer | `256` |
-| Kept routed experts / layer | `144` |
-| Routed MoE layers | `43` |
-| Per-expert params | `25,165,824` |
-| Estimated routed expert params | `155.827B` |
-| Estimated static params | `6.975B` |
-| HF-layout artifact size | ~`88G` |
-| Indexed tensor bytes | `93,684,970,120` |
+## One-command Spark install
 
-The checkpoint uses the compact-K layout:
+Run this on the DGX Spark. `HF_TOKEN` is only required if the model repo is private or not already cached on the machine.
 
-```text
-config.json:            n_routed_experts = 144
-inference/config.json:  n_routed_experts = 144
-inference/config.json:  n_activated_experts = 6
+```bash
+HF_TOKEN=... bash -lc 'set -euo pipefail; cd /home/sero/spark; rm -rf deepseek-v4-flash-spark-200k; git clone https://github.com/0xSero/deepseek-v4-flash-spark-200k.git; cd deepseek-v4-flash-spark-200k; ./install.sh --profile k144-nospec-200k --launch'
 ```
 
-Hash-routed layers use `tid2eid` remapping by gate-weight cosine similarity rather than arbitrary fallback remapping. This avoids the earlier hash-layer route-collapse bug seen in smaller/older REAP builds.
-
-## Observation Source
-
-Expert selection used the available rows from:
-
-[`0xSero/deepseek-v4-flash-reap-observations-v2`](https://huggingface.co/datasets/0xSero/deepseek-v4-flash-reap-observations-v2)
+Do not commit tokens into the repo or a model card. Pass them only through the environment for the one command above.
 
-Snapshot metadata:
+## Exact working profile
 
-```text
-snapshot_label: partial-v2-21289
-available_rows_used: 21289
-all_experts_observed: true
-categories:
-  unicode_stress: 200
-  v2_combined: 21289
-```
-
-## Serving Configuration Tested
-
-Tested on a single B200 using Dockerized vLLM:
+The profile lives at `configs/k144-nospec-200k.env` in the GitHub repo.
 
 ```bash
-VLLM_IMAGE=vllm/vllm-openai:deepseekv4-cu130
-REAP_DIR=/home/ubuntu/ds4-flash-reap/reaps/DeepSeek-V4-Flash-162B-codex-K144-REAP
-SERVED_NAME=ds4-flash-k144-codex-vllm
-MAX_MODEL_LEN=204800
+MODEL_REPO=0xSero/DeepSeek-V4-Flash-162B-codex-K144-REAP
+MODEL_REVISION=d663e8fb16809f6619000648b187b257249ed824
+SERVED_MODEL_NAME=deepseek-v4-flash-k144-g27-cutlass451
+CONTEXT_LENGTH=200000
+KV_CACHE_MEMORY_BYTES=14G
+MAX_NUM_BATCHED_TOKENS=8192
 MAX_NUM_SEQS=1
-MAX_NUM_BATCHED_TOKENS=4096
-GPU_MEMORY_UTILIZATION=0.96
-PORT=8000
+GPU_MEMORY_UTILIZATION=0.88
+WATCHDOG_MIN_AVAILABLE_KB=8388608
+KV_CACHE_DTYPE=fp8
+ENFORCE_EAGER=0
+THINKING=false
+SPECULATIVE_CONFIG=
+VLLM_ENABLE_DEEPSEEK_V4_SPARSE_MLA_WARMUP=0
+VLLM_TRITON_MLA_SPARSE_ALLOW_CUDAGRAPH=1
 ```
 
-Core vLLM flags:
+The launcher enables DeepSeek V4 tokenizer, reasoning parser, tool-call parser, prefix caching, FP8 KV, and CUDA graphs. Do not add `--enforce-eager`; this profile was validated with CUDA graph capture enabled.
 
-```bash
-vllm serve $REAP_DIR \
-  --served-model-name $SERVED_NAME \
-  --trust-remote-code \
-  --kv-cache-dtype fp8 \
-  --block-size 256 \
-  --tensor-parallel-size 1 \
-  --enable-expert-parallel \
-  --gpu-memory-utilization 0.96 \
-  --max-model-len 204800 \
-  --max-num-batched-tokens 4096 \
-  --max-num-seqs 1 \
-  --tokenizer-mode deepseek_v4 \
-  --tool-call-parser deepseek_v4 \
-  --enable-auto-tool-choice \
-  --reasoning-parser deepseek_v4 \
-  --no-enable-flashinfer-autotune
-```
+## Docker runtime
 
-Measured load/runtime notes:
+The expected public image is:
 
 ```text
-vLLM model memory: 86.03 GiB
-server VRAM after warmup: ~176.6 GiB / 183.4 GiB on B200
-advertised max_model_len: 204800
+ghcr.io/0xsero/deepseek-v4-flash-spark-vllm:cutlass451-g27
 ```
 
-## Validation Summary
+The image lineage is the DGX Spark DeepSeek V4 vLLM build `vllm-node-dsv4:latest` with vLLM `0.1.dev17016+g27fd665bd.d20260526` and `nvidia-cutlass-dsl[cu13]==4.5.1`. The installer tags the pulled image as `vllm-node-dsv4-cutlass451:latest`.
 
-Validation run root:
+The repo also carries the runtime patcher used during validation. It applies the nonstandard REAP expert-count router fallback, MXFP4 memory hygiene, optional cute-dsl override hook, and a FlashInfer CUDA IPC `libcudart` fix. It does not modify model weights.
 
-```text
-/home/ubuntu/ds4-flash-reap/runs/k144-bench-20260527T131905Z
-```
+## Validation
 
-Uploaded validation artifacts:
+Validation was run on `spark-2822`, a single DGX Spark / GB10 / SM121 machine, on May 27 2026.
 
 ```text
-validation/200k_smoke.json
-validation/stream_bench.json
-validation/targeted_baseline.compact.json
-validation/targeted_repetition_penalty_1p03.compact.json
+run_dir: /home/sero/spark/benchmarks/deepseek-reap/single-server-sweep/k144-nospec-200k-mnbt8192-20260527T190139Z
+prompt_tokens: 186,390
+TTFT: 345.834 s
+prefill: 538.958 tok/s
+decode: 13.899 tok/s
+needle_retained: true
 ```
 
-### 200K Smoke
-
-| Check | Result | Time | Notes |
-|---|---:|---:|---|
-| Loader sanity | pass | `4.49s` | exact `{"ok": true, "n": 3}` |
-| 200K context echo | pass | `16.78s` | observed `182,974` prompt tokens, exact context JSON |
-| Needle-in-haystack | pass | `7.62s` | exact `BLUE-OWL-38DD5B49D231` |
-
-Overall smoke status: **pass**.
-
-### Stream Bench
+Task coverage at 200K included smoke, ASCII, Unicode, Mermaid, code explanation, religion/philosophy prompts, tool-call fidelity, and a long-needle retrieval test. The 200K sweep completed and retained the needle, but the watchdog logged a low-memory kill at final teardown near the 8 GiB threshold. Treat this as proof that K144 can serve 200K on one Spark, not as the most comfortable always-on daemon profile.
 
-| Shape | TTFT p50 | E2E p50 | Output tok/s p50 | Health |
-|---|---:|---:|---:|---|
-| `latency_1k_out128_c1` | `0.145s` | `1.232s` | `84.38` | no replacement/repetition flags |
-| `decode_1k_out512_c1` | `0.145s` | `1.442s` | `88.07` | no replacement/repetition flags |
-| `prefill_32k_out64_c1` | `0.249s` | `0.845s` | `93.92` | no replacement/repetition flags |
-| `ctx_180k_out64_c1` | `10.222s` | `10.817s` | `95.83` | no replacement/repetition flags |
-| `conc_1k_out128_c4` | `3.246s` | `4.227s` | `86.87` | no replacement/repetition flags |
+K144 MTP2 improved short-context decode in testing, but it was not long-context safe at the tested watchdog thresholds. The published 200K profile is therefore the no-speculative-decoding profile.
 
-### Targeted Formatting / Tool Probe
-
-Baseline targeted probe:
-
-```text
-replacement_ids: []
-health_fail_ids:
-  - ascii_only_bars
-  - mermaid_ascii_stress
-  - unicode_box_request
-  - long_context_visual_recall
-```
-
-Manual inspection showed these were mostly conservative detector triggers on diagram syntax such as repeated `-`, `─`, `|`, and repeated layer blocks, not `U+FFFD` corruption or full sentence-loop collapse.
-
-With `repetition_penalty=1.03`:
-
-```text
-replacement_ids: []
-health_fail_ids:
-  - mermaid_ascii_stress
-  - unicode_box_request
-  - long_context_visual_recall
-```
-
-`ascii_only_bars` passed with `repetition_penalty=1.03`. A stronger penalty (`1.10`) was not recommended because it damaged multilingual/instruction behavior.
-
-## Intended Use
-
-This checkpoint is intended for internal REAP/OPSD validation and serving-envelope experiments:
-
-- long-context serving smoke tests
-- tool/JSON behavior checks
-- Unicode and formatting stress testing
-- comparison against K132/K160 REAP candidates
-- future OPD/OPSD repair experiments
-
-Do not treat this as a final public model release.
-
-## Limitations
-
-- Experimental pruned MoE checkpoint, not a fully validated model
-- Long visual-format generations still need targeted stability work
-- Health detectors are intentionally conservative and can flag valid diagrams
-- Benchmark coverage is smoke/targeted validation, not a complete public benchmark suite
-- This is a derivative of DeepSeek-V4-Flash and inherits base-model licensing/usage constraints
-
-## Provenance Files
-
-Important included files:
-
-```text
-reap_plan.json
-config.json
-inference/config.json
-model.safetensors.index.json
-model-00001-of-00046.safetensors ... model-00046-of-00046.safetensors
-```
+## Intended use
 
-The `reap_plan.json` file records the kept-expert maps, parameter estimate, and observation snapshot.
+This model card is for experimental local inference and reproducibility of the DGX Spark REAP serving recipe. The model is a pruned/quantized DeepSeek V4 Flash derivative; evaluate behavior and license obligations before production use.