Upload 6 files

.gitattributes

@@ -37,3 +37,7 @@ DeepSeek-V4-Flash-0731-IQ2_XS-00001-of-00004.gguf filter=lfs diff=lfs merge=lfs
 DeepSeek-V4-Flash-0731-IQ2_XS-00002-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
 DeepSeek-V4-Flash-0731-IQ2_XS-00003-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
 DeepSeek-V4-Flash-0731-IQ2_XS-00004-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4-Flash-0731-IQ2_XXS-00001-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4-Flash-0731-IQ2_XXS-00002-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4-Flash-0731-IQ2_XXS-00003-of-00004.gguf filter=lfs diff=lfs merge=lfs -text
+DeepSeek-V4-Flash-0731-IQ2_XXS-00004-of-00004.gguf filter=lfs diff=lfs merge=lfs -text

DeepSeek-V4-Flash-0731-IQ2_XXS-00001-of-00004.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b220f41ed3260c709adaab5bd6c9d60f65055d9dc2a5d9522abff081f68163f
+size 27579534528

DeepSeek-V4-Flash-0731-IQ2_XXS-00002-of-00004.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4bba1123d7eabeff65386efed047f8dff6c3c3a0ce223aae2fef4b3739070d2e
+size 27833404672

DeepSeek-V4-Flash-0731-IQ2_XXS-00003-of-00004.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fefe6269b82105de55d2b2f16c2988f1501912c9a2b2c7f64ff86360989e5884
+size 27564969216

DeepSeek-V4-Flash-0731-IQ2_XXS-00004-of-00004.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b1d48df2dcc3944394424c89e531179375960bcdbbdea882711421e57df9ed0
+size 6305127904

README.md

@@ -22,27 +22,30 @@ quantized_by: prometheusAIR
 
 # DeepSeek-V4-Flash-0731 GGUF — fits one RTX PRO 6000 96GB
 
-An imatrix-guided, quality-first GGUF of
-[deepseek-ai/DeepSeek-V4-Flash-0731](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731)
-sized so that **every tensor lives on a single NVIDIA RTX PRO 6000 Blackwell
-96GB GPU** — no CPU expert offload, no second card, no layer spill.
+Two imatrix-guided GGUF variants of
+[deepseek-ai/DeepSeek-V4-Flash-0731](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731),
+both sized so that **every tensor lives on a single NVIDIA RTX PRO 6000
+Blackwell 96GB GPU** — no CPU expert offload, no second card, no layer spill.
 
-**89.9 GiB · ~75 tok/s decode · 384K context on one GPU.**
+- **IQ2_XS quality-first:** 89.9 GiB, approximately 75 tok/s short decode, up to
+  384K context with a practical margin.
+- **IQ2_XXS full-context:** 83.15 GiB, the complete 1,048,576-token context,
+  validated with exact retrieval from a 989,987-token prompt.
 
 ## At a glance
 
-| | |
-|---|---|
-| Base model | `deepseek-ai/DeepSeek-V4-Flash-0731` |
-| Size | 96,556,502,624 bytes — 89.9 GiB, four shards |
-| Recipe | routed-expert gate/up `IQ2_XS` · routed-expert down `IQ3_XXS` · dense/embed/output `Q8_0` |
-| Calibration | model-specific 567K-token corpus, 160 × 2,048-token imatrix chunks |
-| Target GPU | one RTX PRO 6000 Blackwell 96GB (SM 12.0) |
-| Offload | none — weights, KV cache and compute buffers all on GPU |
-| Decode | ~75 tok/s (vs ~25 tok/s for the unquantized-experts master) |
-| Max context on one card | 384K comfortably, 480K at the hard edge |
-| Speculative decoding | DSpark/MTP draft stages not included |
-| License | MIT, inherited from the base model |
+| | IQ2_XS quality-first | IQ2_XXS full-context |
+|---|---|---|
+| Size | 96,556,502,624 bytes / 89.925 GiB | 89,283,036,320 bytes / 83.151 GiB |
+| Expert gate/up | `IQ2_XS` | `IQ2_XXS` |
+| Expert down | `IQ3_XXS` | `IQ3_XXS` |
+| Dense path | `Q8_0` | `Q6_K` |
+| Embedding/output | `Q8_0` | `Q8_0` |
+| Calibration processed | 327,680 tokens at 2K context | 606,208 tokens at 8K context; 100% final expert-slice coverage |
+| Recommended context | 64K default; 384K high-context | Full 1,048,576 tokens |
+| CPU/second-GPU offload | none | none |
+| DSpark/MTP draft stages | excluded | excluded |
+| License | MIT | MIT |
 
 ## Will this run on my GPU?
 
@@ -51,10 +54,20 @@ sized so that **every tensor lives on a single NVIDIA RTX PRO 6000 Blackwell
 | RTX PRO 6000 Blackwell 96GB (Workstation Edition) | Yes — this is the tested configuration |
 | Two smaller cards totalling ≥ 96 GB | Should work with `--split-mode layer`. Untested here |
 | One card under 96 GB | Needs `-ncmoe N` to push expert layers to CPU, which costs a lot of speed. Untested here |
-| A 96 GB card of another architecture | Requires working `IQ2_XS` / `IQ3_XXS` kernels for that arch. Untested here |
+| A 96 GB card of another architecture | Requires working `IQ2_XS` / `IQ2_XXS` / `IQ3_XXS` / `Q6_K` kernels as applicable. Untested here |
 
 Leave roughly 2–3 GB of the card free for your display/compositor if the GPU
-is also driving a monitor — the numbers below assume a headless card.
+is also driving a monitor. The full-context variant left 1,752 MiB in the tested
+headless configuration, so a display workload can make 1M impractical even on
+the same card.
+
+### Which variant should I use?
+
+- Choose `IQ2_XS` for the stronger weight recipe, approximately 75 tok/s short
+  decode, and contexts through the recommended 384K profile.
+- Choose `IQ2_XXS` only when you need more than the `IQ2_XS` artifact can
+  allocate. It reaches the full 1M context by lowering gate/up experts and dense
+  matrices, and long prompts become much slower to prefill.
 
 ## Quick start
 
@@ -88,6 +101,23 @@ CUDA_VISIBLE_DEVICES=0 llama-server \
 
 Do not use ubatch 2,048 at 384K on a 96 GB card — see the table below.
 
+### Full 1M profile
+
+```bash
+CUDA_VISIBLE_DEVICES=0 llama-server \
+  --model DeepSeek-V4-Flash-0731-IQ2_XXS-00001-of-00004.gguf \
+  --host 127.0.0.1 --port 8080 \
+  --device CUDA0 --split-mode none -ngl 999 --fit off \
+  --ctx-size 1048576 \
+  -b 2048 -ub 1024 \
+  -np 1 -fa on --jinja \
+  --reasoning-budget 4096 \
+  --reasoning-budget-message "I am out of thinking time. I will now stop deliberating and give my best final answer directly."
+```
+
+This profile uses the `IQ2_XXS` shards, not the larger `IQ2_XS` files. Keep
+ubatch at 1,024 and one parallel slot unless you remeasure the entire allocation.
+
 ## Measured VRAM
 
 Measured on a card reporting 97,887 MiB total, CUDA device 0 only, one slot,
@@ -111,14 +141,32 @@ where the wall is; it is not a deployment recommendation. Driver version,
 display usage, CUDA version and llama.cpp revision all shift these numbers, so
 treat them as a starting point and verify on your own card.
 
-These are allocation and generation checks. They do **not** establish retrieval
-quality at these depths — see limitations.
-
-> **In progress — full 1M context.** A smaller `IQ2_XXS` build is being prepared
-> for this repo, targeting the model's full 1,048,576-token context on the same
-> single card: expert gate/up `IQ2_XXS`, expert down `IQ3_XXS`, dense `Q6_K`,
-> embedding/output `Q8_0`, `F16` KV cache. The extra context comes from
-> shrinking the weights, not from quantizing the KV cache.
+These are allocation and generation checks for the `IQ2_XS` variant. They do
+**not** establish retrieval quality at every listed depth.
+
+### IQ2_XXS full-context measurements
+
+The full-context artifact loaded at `--ctx-size 1048576`, `-b 2048`,
+`-ub 1024`, and `-np 1`, with every model tensor on CUDA0:
+
+| Measurement | Result |
+|---|---:|
+| Total CUDA0 use after startup/warmup | 95,409 MiB |
+| Highest CUDA0 use during tested prefills | 95,537 MiB |
+| Minimum free CUDA0 VRAM | 1,752 MiB |
+| CUDA1 model allocation | zero |
+| 6,010-token retrieval | exact; 1,746 tok/s prefill |
+| 130,999-token retrieval | exact; 1,101 tok/s prefill |
+| 989,987-token retrieval | exact at 97% document depth; 308.56 tok/s average prefill |
+| Near-1M decode | 25.44 tok/s |
+| Near-1M prefill wall time | 3,208.36 seconds / 53.47 minutes |
+
+The final test processed 989,987 prompt tokens and 990,079 total tokens, then
+returned `OBSIDIAN-LANTERN-ONE-MILLION` exactly from record 40,000 of 41,248.
+The context allocation did not grow beyond the measured 95,537 MiB peak. This
+proves startup, warmup, meaningful prefill, generation, and retrieval near the
+model's configured limit on the tested runtime; it is not a broad long-context
+benchmark.
 
 ## Reasoning, tools, and the empty-response trap
 
@@ -171,35 +219,50 @@ agentic use. Tool calls and tool-result continuation were both validated here.
 ## Requirements
 
 - A CUDA llama.cpp build with DeepSeek-V4 support and working SM 12.0 kernels
-  for `IQ2_XS`, `IQ3_XXS` and `Q8_0`. Validated on build `b10148` (`1476ef0e5`).
+  for `IQ2_XS`, `IQ2_XXS`, `IQ3_XXS`, `Q6_K`, and `Q8_0`. Validated on
+  build `b10148` (`1476ef0e5`).
 - `--jinja`, so the embedded chat template is used.
 
 Older builds may not know the architecture, its metadata, or the template.
 
 ## What's in the file
 
+### IQ2_XS quality-first tensor census
+
 | Tensor group | Count | Type |
 |---|---:|---|
 | Routed expert gate/up | 86 | `IQ2_XS` |
 | Routed expert down | 43 | `IQ3_XXS` |
 | Dense matrices, embedding, output | 661 | `Q8_0` |
-| Routers | 43 | `BF16` (source precision retained) |
+| Routers | 43 | `BF16` |
+| Norms, biases, tables | 492 | `F32` |
+| Routing/index tables | 3 | `I32` |
+| **Total** | **1,328** | |
+
+### IQ2_XXS full-context tensor census
+
+| Tensor group | Count | Type |
+|---|---:|---|
+| Routed expert gate/up | 86 | `IQ2_XXS` |
+| Routed expert down | 43 | `IQ3_XXS` |
+| Eligible dense matrices | 659 | `Q6_K` |
+| Token embedding and output | 2 | `Q8_0` |
+| Routers | 43 | `BF16` |
 | Norms, biases, tables | 492 | `F32` |
 | Routing/index tables | 3 | `I32` |
 | **Total** | **1,328** | |
 
 Routed experts dominate this checkpoint's storage, so quantizing only the dense
-layers cannot reach 96 GB. This recipe spends the bits where they buy the most:
-expert down-projections get an extra bit over gate/up, and the entire dense
-path, embedding and output stay at `Q8_0`.
+layers cannot reach 96 GB. Both recipes spend more bits on expert down
+projections than gate/up. The full-context variant additionally moves eligible
+dense matrices from Q8_0 to Q6_K while retaining embedding and output at Q8_0.
 
-> **Note on the displayed quant type.** The GGUF header records
-> `general.file_type = Q8_0`, because `Q8_0` was the base type passed to
-> `llama-quantize` and the expert types were applied as per-tensor overrides.
-> Tools that read that single field — including some Hub viewers — may
-> therefore label this file "Q8_0". The table above is what is actually in it.
+> **Note on the displayed quant type.** These are per-tensor mixed recipes. The
+> GGUF `general.file_type` records only the base type passed to `llama-quantize`,
+> so Hub viewers may show `Q8_0` for the IQ2_XS files or `Q6_K` for the IQ2_XXS
+> files. The tensor censuses above are authoritative.
 
-**No `IQ1_S`, `IQ2_S` or `IQ3_S` tensors are present.** Those three types have
+**No `IQ1_S`, `IQ2_S`, or `IQ3_S` tensors are present.** Those three types have
 broken CUDA kernels on SM 12.0, which silently poisons many otherwise-plausible
 sub-4-bit quants on Blackwell. All 43 layers, the release compression-ratio
 metadata, and the validated 0731 chat template are intact.
@@ -224,6 +287,14 @@ official 0731 encoding fixtures. Processed as 160 chunks × 2,048 tokens into 76
 imatrix entries; lowest final sparse routed-expert coverage 99.61%. Corpus
 SHA-256 `401ea959421622e148d062d4c9719d4c31027f4af1ee10ebff0525e4bb0a13ea`.
 
+The `IQ2_XXS` variant used a separate expanded corpus with 613,766 tokenizer
+tokens across 23 sections. `llama-imatrix` processed 74 chunks × 8,192 tokens,
+or 606,208 tokens, into 769 entries. Direct inspection of the final imatrix
+found all 256 slices populated for every routed-expert matrix: 100.00% final
+slice coverage. Corpus SHA-256:
+`0b0503b27680df69a8f40507cb329bc49a0bb3cd8c7f30fa4da1fde9ce8cc3bc`.
+Quantization took 5,323.19 seconds (88.72 minutes) with 12 threads.
+
 The embedded Jinja template was validated byte-for-byte against the official
 0731 encoder fixtures, including tool calls and tool-result continuation.
 
@@ -237,11 +308,19 @@ Focused smoke tests, run against both this quant and the fidelity master:
 - DSML tool call plus tool-result continuation
 - a generated Python `merge_intervals`, passing 4/4 executable cases
 
-Decode clustered at ~75 tok/s (63–77 across individual tests; the low outlier
-was a 21-token generation where fixed overhead dominates), against 21–26 tok/s
-for the partially-offloaded fidelity master on identical prompts — about 3×.
+Decode for the IQ2_XS variant clustered at approximately 75 tok/s (63–77
+across individual tests; the low outlier was a 21-token generation where fixed
+overhead dominates), against 21–26 tok/s for the partially-offloaded fidelity
+master on identical prompts — about 3×.
+
+The IQ2_XXS variant passed the same exact-output, arithmetic, executable coding,
+Chinese, DSML tool-call, and tool-continuation checks. High-reasoning arithmetic
+and coding hit an old 768-token smoke cap before emitting content, but both
+completed correctly with the documented 4,096-token budget. Its generated
+`merge_intervals` passed 4/4 executable cases. Retrieval was exact at 6K, 131K,
+and 989,987 prompt tokens; the near-1M needle was at 97% document depth.
 
-**This is an implementation smoke test, not a benchmark suite, and not evidence
+**These are implementation smoke tests, not a benchmark suite, and not evidence
 of quality parity with the released checkpoint.**
 
 ## Limitations
@@ -249,8 +328,13 @@ of quality parity with the released checkpoint.**
 - Low-bit expert quantization can change reasoning, coding, factuality, routing
   and tool behaviour in ways short smoke tests will not surface. Evaluate on
   your own workload before relying on it.
-- Long-context *retrieval* quality was not validated. The context table proves
-  allocation, not recall at depth.
+- The near-1M check is a single synthetic needle test, not evidence of broad
+  million-token reasoning, multi-hop retrieval, or benchmark quality.
+- A near-1M prefill took approximately 53.5 minutes on the tested host.
+- The full-context configuration retained only 1,752 MiB at its measured peak;
+  display use and runtime/driver differences can eliminate that margin.
+- High-reasoning requests may consume more than 768 completion tokens before
+  emitting final content; provide a realistic `max_tokens` budget.
 - No DSpark speculative draft stages.
 - No guarantees for other GPUs or other llama.cpp revisions.
 
@@ -260,6 +344,8 @@ appropriate to your application.
 
 ## Files
 
+### IQ2_XS quality-first
+
 | File | Size | SHA-256 |
 |---|---:|---|
 | `DeepSeek-V4-Flash-0731-IQ2_XS-00001-of-00004.gguf` | 27.50 GiB | `b09025603eb0192bb5dae42084c096427813658f5be0b7ec19341a96b3ce7e20` |
@@ -267,6 +353,15 @@ appropriate to your application.
 | `DeepSeek-V4-Flash-0731-IQ2_XS-00003-of-00004.gguf` | 27.48 GiB | `8c7dd99da2d54f39660659d8c401811149bf341c307fedb30a19f510299e2495` |
 | `DeepSeek-V4-Flash-0731-IQ2_XS-00004-of-00004.gguf` | 7.27 GiB | `eb0cccbdf4b94ecf42d408b2d03b5a0bb3d706a6f0ca0f63ac0256f947a08f94` |
 
+### IQ2_XXS full-context
+
+| File | Size | SHA-256 |
+|---|---:|---|
+| `DeepSeek-V4-Flash-0731-IQ2_XXS-00001-of-00004.gguf` | 25.685443 GiB | `0b220f41ed3260c709adaab5bd6c9d60f65055d9dc2a5d9522abff081f68163f` |
+| `DeepSeek-V4-Flash-0731-IQ2_XXS-00002-of-00004.gguf` | 25.921878 GiB | `4bba1123d7eabeff65386efed047f8dff6c3c3a0ce223aae2fef4b3739070d2e` |
+| `DeepSeek-V4-Flash-0731-IQ2_XXS-00003-of-00004.gguf` | 25.671878 GiB | `fefe6269b82105de55d2b2f16c2988f1501912c9a2b2c7f64ff86360989e5884` |
+| `DeepSeek-V4-Flash-0731-IQ2_XXS-00004-of-00004.gguf` | 5.872108 GiB | `5b1d48df2dcc3944394424c89e531179375960bcdbbdea882711421e57df9ed0` |
+
 ```bash
 sha256sum -c SHA256SUMS
 ```

SHA256SUMS

@@ -2,3 +2,7 @@ b09025603eb0192bb5dae42084c096427813658f5be0b7ec19341a96b3ce7e20  DeepSeek-V4-Fl
 faf2467279b673bbcdfea7c8413f762f39973b3a758808ec2d6bbdd008710f5f  DeepSeek-V4-Flash-0731-IQ2_XS-00002-of-00004.gguf
 8c7dd99da2d54f39660659d8c401811149bf341c307fedb30a19f510299e2495  DeepSeek-V4-Flash-0731-IQ2_XS-00003-of-00004.gguf
 eb0cccbdf4b94ecf42d408b2d03b5a0bb3d706a6f0ca0f63ac0256f947a08f94  DeepSeek-V4-Flash-0731-IQ2_XS-00004-of-00004.gguf
+0b220f41ed3260c709adaab5bd6c9d60f65055d9dc2a5d9522abff081f68163f  DeepSeek-V4-Flash-0731-IQ2_XXS-00001-of-00004.gguf
+4bba1123d7eabeff65386efed047f8dff6c3c3a0ce223aae2fef4b3739070d2e  DeepSeek-V4-Flash-0731-IQ2_XXS-00002-of-00004.gguf
+fefe6269b82105de55d2b2f16c2988f1501912c9a2b2c7f64ff86360989e5884  DeepSeek-V4-Flash-0731-IQ2_XXS-00003-of-00004.gguf
+5b1d48df2dcc3944394424c89e531179375960bcdbbdea882711421e57df9ed0  DeepSeek-V4-Flash-0731-IQ2_XXS-00004-of-00004.gguf