Document DGX Spark and DFlash benchmark results

README.md

@@ -26,10 +26,11 @@ datasets:
 # llm-jp-4-8b-instruct — NVFP4 (Japanese-calibrated)
 
 NVFP4 quantization of [llm-jp/llm-jp-4-8b-instruct](https://huggingface.co/llm-jp/llm-jp-4-8b-instruct)
-for native serving on NVIDIA Blackwell GPUs (SM120/SM121) with vLLM — **8.9 GB
-(0.52× of BF16), 1.76× single-stream decode speedup, ~99.1% quality
-recovery**, calibrated on Japanese chat/instruction data through the model's
-own Harmony chat template.
+for native serving on NVIDIA Blackwell GPUs with vLLM: RTX PRO 6000 (SM120)
+and DGX Spark / GB10 (SM121). It is **8.9 GB (0.52× of BF16)**, recovers
+**~99.1%** of the BF16 quality score, and improves single-stream decode by
+**1.76× on RTX PRO 6000** and **2.02× on DGX Spark / GB10** in the
+llama-benchy measurements below.
 
 The calibration set is a Japanese-heavy chat/instruction mix matched to the
 model's expected deployment distribution. The full recipe, calibration
@@ -112,18 +113,50 @@ all rounds of both models in one batch session, generation max_tokens=2048.
 Round-to-round generation spread is ±0.04–0.12. Scores are **relative**
 comparisons only — not comparable to LLM-jp's published GPT-judge numbers.
 
-## Throughput (RTX PRO 6000, pp=2048/tg=256, llama-benchy, vLLM 0.24)
+## Throughput: BF16 vs NVFP4
 
-| concurrency | BF16 decode tok/s | NVFP4 decode tok/s | speedup | TTFT (e2e) |
-|---|---|---|---|---|
-| 1 | 84.4 | 148.1 | **1.76×** | 155→91 ms |
-| 4 | 297 | 368 | 1.24× | 438→238 ms |
-| 8 | 450 | 833 | 1.85× | 847→439 ms |
-| 16 | 801 | 1170 | 1.46× | 1627→808 ms |
+Measured with llama-benchy, vLLM 0.24.0, 2,048 prompt tokens, 256 generated
+tokens, and three measured runs.
 
-c=1 decode is stable to <0.1% across runs; the c=4–16 points carry ±10–15%
-run-to-run spread (3 runs each, means shown). KV-cache capacity at
+| Hardware | SM | concurrency | BF16 decode tok/s | NVFP4 decode tok/s | speedup | TTFT (e2e) |
+|---|---:|---:|---:|---:|---:|---:|
+| RTX PRO 6000 | SM120 | 1 | 84.4 | 148.1 | **1.76×** | 155→91 ms |
+| RTX PRO 6000 | SM120 | 4 | 297 | 368 | 1.24× | 438→238 ms |
+| RTX PRO 6000 | SM120 | 8 | 450 | 833 | 1.85× | 847→439 ms |
+| RTX PRO 6000 | SM120 | 16 | 801 | 1170 | 1.46× | 1627→808 ms |
+| DGX Spark / GB10 | SM121 | 1 | 13.80 | 27.84 | **2.02×** | 648→348 ms |
+| DGX Spark / GB10 | SM121 | 4 | 56.49 | 106.13 | 1.88× | 1775→1084 ms |
+| DGX Spark / GB10 | SM121 | 8 | 97.36 | 173.16 | 1.78× | 2900→1597 ms |
+| DGX Spark / GB10 | SM121 | 16 | 153.11 | 253.57 | 1.66× | 4763→2638 ms |
+
+On RTX PRO 6000, c=1 decode is stable to <0.1% across runs; the c=4–16
+points carry ±10–15% run-to-run spread. KV-cache capacity at
 `--gpu-memory-utilization 0.9` grows ~15% thanks to the smaller weights.
+The DGX Spark / GB10 rows were measured locally on 2026-07-26 against BF16
+`llm-jp/llm-jp-4-8b-instruct` revision
+`098f2b2cf33021eba19a6d3582aa3d071ccc0aff` and this NVFP4 checkpoint revision
+`748431048bb5b7114424ae6c0a17bdd2172cc9cf`.
+
+## Compatible DFlash Drafter
+
+A companion DFlash drafter is available at
+[`kel-jp/llm-jp-4-8b-instruct-NVFP4-speculator.dflash`](https://huggingface.co/kel-jp/llm-jp-4-8b-instruct-NVFP4-speculator.dflash).
+
+It is not a standalone text-generation model. Use it as a draft/speculator
+model while serving this NVFP4 model as the verifier in stock vLLM:
+
+```bash
+vllm serve kel-jp/llm-jp-4-8b-instruct-NVFP4 \
+  --trust-remote-code \
+  --reasoning-parser llmjp4 \
+  --speculative-config '{"method":"dflash","model":"kel-jp/llm-jp-4-8b-instruct-NVFP4-speculator.dflash","num_speculative_tokens":3}'
+```
+
+On DGX Spark / GB10 (SM121), a Japanese ELYZA-100 streaming decode benchmark
+measured 14.42 tok/s for BF16, 29.73 tok/s for this NVFP4 checkpoint, and
+49.04 tok/s for NVFP4 + DFlash. That is a 3.40× mean decode throughput
+increase from BF16 to NVFP4 + DFlash, with DFlash adding 1.64× over the NVFP4
+verifier baseline in the paired run.
 
 ## Serving notes
 
@@ -134,8 +167,7 @@ benchmark-validated version.** 0.25.0 loads and scores identically in our
 testing (same kernels, +2% decode), but we observed one unreproduced
 `cudaErrorIllegalAddress` engine crash under high-concurrency load on it and
 haven't cleared it for production yet. Validated end-to-end on **SM120**
-(RTX PRO 6000); **SM121 (DGX Spark / GB10) validation is pending** and this
-card will be updated once verified.
+(RTX PRO 6000) and **SM121** (DGX Spark / GB10).
 
 The base model emits its responses in OpenAI-Harmony structure
 (`<|channel|>analysis/final<|message|>…`), which is why the bundled plugin is