jackasda211233 commited on
Commit
febbfe1
·
verified ·
1 Parent(s): a2a22dc

Document Fable Fusion 711 MTP Q8_0 comparison

Browse files
Files changed (1) hide show
  1. README.md +25 -0
README.md CHANGED
@@ -288,6 +288,31 @@ A 15-case behaviour rubric (action-first style + coding discipline + held-out ge
288
 
289
  PatchCode scores higher while emitting ~⅓ the tokens — the base rambled (~311 tokens of hedging preamble), PatchCode was terse and on-target. λ=0.5 is the sweet spot: higher strengths also got terse but fell *below* the base (an over-loud LoRA delta hurting calibrated behaviour). Caveat: a behaviour rubric, not a multi-turn agent turn-count; single-temperature, small per-category N.
290
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
291
  ### Why there is no Q8 release
292
 
293
  A near-lossless `Q8_0` was built and tested 5-seed head-to-head against the shipped IQ4_NL (table 2). It showed **no beyond-noise edge on any axis** and is ~2× the size — near-lossless precision buys nothing measurable here because the build is ceiling-limited and noisy, not precision-limited. Attention-promotion mixed recipes (c76 and the overnight precision×promotion matrix) were tested for the same reason and ruled out: promotion destroyed discipline for no build gain. Only `IQ4_NL` and `BF16` are released.
 
288
 
289
  PatchCode scores higher while emitting ~⅓ the tokens — the base rambled (~311 tokens of hedging preamble), PatchCode was terse and on-target. λ=0.5 is the sweet spot: higher strengths also got terse but fell *below* the base (an over-loud LoRA delta hurting calibrated behaviour). Caveat: a behaviour rubric, not a multi-turn agent turn-count; single-temperature, small per-category N.
290
 
291
+ ### 4 — Cross-model practical comparison: Fable Fusion 711 MTP Q8_0
292
+
293
+ We also replayed the hardened KritaForge and GameForge coding-agent builds with the external model [`DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF`](https://huggingface.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF), specifically:
294
+
295
+ `Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-MTP-Q8_0.gguf`
296
+
297
+ This identifies the exact DavidAU repository and file tested; DavidAU publishes several related Qwen3.6 models and both MTP and non-MTP quants in this repository.
298
+
299
+ | practical build | Fable Fusion 711 MTP Q8_0 (5-seed) | PatchCode IQ4_NL (stored) | directional delta |
300
+ |---|---:|---:|---:|
301
+ | KritaForge (33 checks) | `0.806` mean · `0.818` median · `0.758–0.818` range | `0.788` (26/33) | `+0.018` |
302
+ | GameForge (54 checks) | `0.859` mean · `0.852` median · `0.815–0.907` range | `0.907` (49/54) | `−0.048` |
303
+
304
+ **Read:** these results do not establish a winner. Fable Fusion 711 was marginally ahead on KritaForge and PatchCode was marginally ahead on GameForge, but both gaps are below `0.05`. The stored PatchCode values are single-run baselines, so differences this small are directional only.
305
+
306
+ Known limits:
307
+
308
+ - This is **not quantization-matched**: the external model was its MTP `Q8_0` file (30.24 GB), while PatchCode was the shipped `IQ4_NL` file (about 16.6 GB).
309
+ - Only Fable Fusion 711 was newly run at five seeds. PatchCode was not rerun; its values come from prior on-disk objective-verifier receipts using the same tasks.
310
+ - The task prompts, component verifiers, agent interface, 160k context, and sampling settings were matched. The model-specific loader and embedded chat template necessarily differed.
311
+ - The runtime loaded the base Q8 model but ignored the extra MTP draft tensors, so this evaluates generated capability, not MTP speculative-decoding speed.
312
+ - These are practical coding-agent builds, not claims about general intelligence, creativity, safety, or the broader capabilities described on the DavidAU model card.
313
+
314
+ Next comparison step: rerun both models under the same quantization class and collect a multi-seed PatchCode baseline before treating a sub-`0.05` gap as meaningful.
315
+
316
  ### Why there is no Q8 release
317
 
318
  A near-lossless `Q8_0` was built and tested 5-seed head-to-head against the shipped IQ4_NL (table 2). It showed **no beyond-noise edge on any axis** and is ~2× the size — near-lossless precision buys nothing measurable here because the build is ceiling-limited and noisy, not precision-limited. Attention-promotion mixed recipes (c76 and the overnight precision×promotion matrix) were tested for the same reason and ruled out: promotion destroyed discipline for no build gain. Only `IQ4_NL` and `BF16` are released.