Card: controlled test — f16 MTP head is a wash (~0.83 both); mtpF16 redundant, use base
Browse files
README.md
CHANGED
|
@@ -79,19 +79,19 @@ the **output head** (lm-head) and the **MTP draft head**:
|
|
| 79 |
|---|---|---|---|---|
|
| 80 |
| `…-STRIX-embQ8.gguf` | 21.8 GB | 4-bit | 4-bit (Qwopus donor) | **fastest** — the recommended default |
|
| 81 |
| `…-STRIX-embQ8-headQ6.gguf` | 22.2 GB | **Q6_K** | 4-bit (Qwopus donor) | better output quality |
|
| 82 |
-
| `…-STRIX-embQ8-mtpF16.gguf` | ~22.4 GB | 4-bit | **f16** (BF16 base-27B donor) |
|
| 83 |
|
| 84 |
- **Output head** (`output.weight`): 4-bit vs `Q6_K`. Q6_K is a touch more faithful at ~5–7% slower
|
| 85 |
decode (trade measured on the [27B sibling](https://huggingface.co/plunderstruck/Qwen3.6-27B-MTP-ROCmFP4-GGUF)).
|
| 86 |
- **MTP draft head** (`blk.96`): the base & headQ6 variants graft a **4-bit** head from a **Qwopus-27B**
|
| 87 |
donor (a Claude-Opus distill); the **mtpF16** variant keeps the head at **f16** from a **BF16 base-27B**
|
| 88 |
donor — mirroring [PiehSoft's high-precision-head choice](https://huggingface.co/PiehSoft/Qwen3.6-40B-Deckard-MTP-Q6_K).
|
| 89 |
-
**
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
- **Q8 embeddings** (not f16): the source is Q8_0, so f16 would be fake-f16 bloat — Q8 matches the
|
| 96 |
source precision exactly.
|
| 97 |
|
|
|
|
| 79 |
|---|---|---|---|---|
|
| 80 |
| `…-STRIX-embQ8.gguf` | 21.8 GB | 4-bit | 4-bit (Qwopus donor) | **fastest** — the recommended default |
|
| 81 |
| `…-STRIX-embQ8-headQ6.gguf` | 22.2 GB | **Q6_K** | 4-bit (Qwopus donor) | better output quality |
|
| 82 |
+
| `…-STRIX-embQ8-mtpF16.gguf` | ~22.4 GB | 4-bit | **f16** (BF16 base-27B donor) | experiment — no acceptance gain vs base |
|
| 83 |
|
| 84 |
- **Output head** (`output.weight`): 4-bit vs `Q6_K`. Q6_K is a touch more faithful at ~5–7% slower
|
| 85 |
decode (trade measured on the [27B sibling](https://huggingface.co/plunderstruck/Qwen3.6-27B-MTP-ROCmFP4-GGUF)).
|
| 86 |
- **MTP draft head** (`blk.96`): the base & headQ6 variants graft a **4-bit** head from a **Qwopus-27B**
|
| 87 |
donor (a Claude-Opus distill); the **mtpF16** variant keeps the head at **f16** from a **BF16 base-27B**
|
| 88 |
donor — mirroring [PiehSoft's high-precision-head choice](https://huggingface.co/PiehSoft/Qwen3.6-40B-Deckard-MTP-Q6_K).
|
| 89 |
+
**Controlled warm comparison (identical 6-turn session, f16/f16 KV both):** 4-bit/Qwopus head **~0.82
|
| 90 |
+
sustained**, f16/base-27B head **~0.84 sustained** — a **wash** (the gap is within turn-to-turn noise; the
|
| 91 |
+
earlier 0.95-vs-0.77 split was just cold-vs-cold noise). **Conclusion: the f16 MTP head does not
|
| 92 |
+
meaningfully improve acceptance** — neither head precision nor donor moved it. So **`mtpF16` is effectively
|
| 93 |
+
redundant with `base`** (same speed, ~0.6 GB bigger). **Use `base` (or `headQ6` for output quality);
|
| 94 |
+
`mtpF16` is kept only as a documented negative-result experiment.**
|
| 95 |
- **Q8 embeddings** (not f16): the source is Q8_0, so f16 would be fake-f16 bloat — Q8 matches the
|
| 96 |
source precision exactly.
|
| 97 |
|