I've been running this finetune (IQ4_XS) as my daily driver on an rtx5090 with llama.cpp + --spec-type ngram-mod, which gets ~2.5x peak / ~1.7x session-average TG over the ~77 t/s no-spec baseline on long-context workloads. Wanted to ask about MTP.
The runtime side is now in place upstream: PR #22673 adds MTP draft support to llama.cpp (tested explicitly on Qwen3.6 27B). I've built it against CUDA 13 and have --spec-type mtp working end-to-end against froggeric/Qwen3.6-27B-MTP-GGUF Q4_K_M at 262k context. On my own bench (q8_0 K / turbo3 V KV cache, MTP draft depth 3) I'm seeing ~115-138 t/s decode on novel generation (code synthesis, explanatory chat) - exactly the workload where ngram-mod degrades to ~baseline. The base-to-MTP-GGUF conversion path is also clearly established (froggeric, RDson, and havenoammo have all published MTP GGUFs of stock Qwen3.6 27B).
**So the question: can the Heretic abliteration + Unsloth recovery + NEO-CODE Di-IMatrix MAX pipeline be re-run on the MTP-equipped Qwen3.6 27B base in a way that preserves the MTP heads through to the final GGUF?
The reason it would matter on this particular finetune: ngram-mod is great on doc-quote / IDE-context workloads but degrades to ~1.0x on novel generation (chat, free-form code), which is most of the interactive workload. MTP covers exactly that gap.
Happy to test any candidate build you produce against the same runtime.
Currently the MTP layers are not in the heretic (and later fine tune version) and GGUFs.
Part of this is because Heretic (and tuning) excludes/ignores these layers. In fact if you load the 27B for tuning (root, core model from Qwen WITH MTP layers/tensors ) the "tuned" version will not tune or have the MTP layers.
These have the be re-merged with the safe tensor files manually [and model index].
The issue is this however:
These layers/tensors (MTP) would not have been heretic'ed OR tuned ; resulting in possible performance / "censored" issues occurring.
Although I can add these MTP tensors/layers back to the final model, these are the core reasons it was not done.
@DavidAUi think you still should add the original MTP layers back in, they still improve speed on more predictable content ie code. someone made an MTP version of your 40B by adding it back from the 27B and it does improve speed. it will however not improve speeds on uncensored outputs, but for general purpose it's still useful.
