plunderstruck commited on
Commit
4652708
·
verified ·
1 Parent(s): 9677f59

Card: controlled test — f16 MTP head is a wash (~0.83 both); mtpF16 redundant, use base

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -79,19 +79,19 @@ the **output head** (lm-head) and the **MTP draft head**:
79
  |---|---|---|---|---|
80
  | `…-STRIX-embQ8.gguf` | 21.8 GB | 4-bit | 4-bit (Qwopus donor) | **fastest** — the recommended default |
81
  | `…-STRIX-embQ8-headQ6.gguf` | 22.2 GB | **Q6_K** | 4-bit (Qwopus donor) | better output quality |
82
- | `…-STRIX-embQ8-mtpF16.gguf` | ~22.4 GB | 4-bit | **f16** (BF16 base-27B donor) | experimental high-precision head |
83
 
84
  - **Output head** (`output.weight`): 4-bit vs `Q6_K`. Q6_K is a touch more faithful at ~5–7% slower
85
  decode (trade measured on the [27B sibling](https://huggingface.co/plunderstruck/Qwen3.6-27B-MTP-ROCmFP4-GGUF)).
86
  - **MTP draft head** (`blk.96`): the base & headQ6 variants graft a **4-bit** head from a **Qwopus-27B**
87
  donor (a Claude-Opus distill); the **mtpF16** variant keeps the head at **f16** from a **BF16 base-27B**
88
  donor — mirroring [PiehSoft's high-precision-head choice](https://huggingface.co/PiehSoft/Qwen3.6-40B-Deckard-MTP-Q6_K).
89
- **Measured MTP acceptance (mtpF16):** ~0.77 cold, but **~0.80–0.89 sustained (avg ~0.85)** across a warm
90
- 6-turn coding sessionright in PiehSoft's reported BF16-head range (85–100% sustained). The
91
- 4-bit/Qwopus-donor head hit ~0.95 on a single short test, but there's **no warm multi-turn number for it,
92
- so this is not a controlled comparison** — both heads are solid (~0.8–0.95). Pick **mtpF16** for the
93
- PiehSoft-style high-precision head, or **base** for max speed/simplicity. (Earlier notes called mtpF16 an
94
- underperformer based only on its *cold* number; the warm sustained acceptance corrects that.)
95
  - **Q8 embeddings** (not f16): the source is Q8_0, so f16 would be fake-f16 bloat — Q8 matches the
96
  source precision exactly.
97
 
 
79
  |---|---|---|---|---|
80
  | `…-STRIX-embQ8.gguf` | 21.8 GB | 4-bit | 4-bit (Qwopus donor) | **fastest** — the recommended default |
81
  | `…-STRIX-embQ8-headQ6.gguf` | 22.2 GB | **Q6_K** | 4-bit (Qwopus donor) | better output quality |
82
+ | `…-STRIX-embQ8-mtpF16.gguf` | ~22.4 GB | 4-bit | **f16** (BF16 base-27B donor) | experiment no acceptance gain vs base |
83
 
84
  - **Output head** (`output.weight`): 4-bit vs `Q6_K`. Q6_K is a touch more faithful at ~5–7% slower
85
  decode (trade measured on the [27B sibling](https://huggingface.co/plunderstruck/Qwen3.6-27B-MTP-ROCmFP4-GGUF)).
86
  - **MTP draft head** (`blk.96`): the base & headQ6 variants graft a **4-bit** head from a **Qwopus-27B**
87
  donor (a Claude-Opus distill); the **mtpF16** variant keeps the head at **f16** from a **BF16 base-27B**
88
  donor — mirroring [PiehSoft's high-precision-head choice](https://huggingface.co/PiehSoft/Qwen3.6-40B-Deckard-MTP-Q6_K).
89
+ **Controlled warm comparison (identical 6-turn session, f16/f16 KV both):** 4-bit/Qwopus head **~0.82
90
+ sustained**, f16/base-27B head **~0.84 sustained** a **wash** (the gap is within turn-to-turn noise; the
91
+ earlier 0.95-vs-0.77 split was just cold-vs-cold noise). **Conclusion: the f16 MTP head does not
92
+ meaningfully improve acceptance** — neither head precision nor donor moved it. So **`mtpF16` is effectively
93
+ redundant with `base`** (same speed, ~0.6 GB bigger). **Use `base` (or `headQ6` for output quality);
94
+ `mtpF16` is kept only as a documented negative-result experiment.**
95
  - **Q8 embeddings** (not f16): the source is Q8_0, so f16 would be fake-f16 bloat — Q8 matches the
96
  source precision exactly.
97