Initital Tests, Magnitude Blowout, T13 Restoration, and Magistral's Thinking Layer
It could be recency bias making me forget how good all the other 24B models are, but in my latest tests, this merge outperformed several others that were similar, yet either produced broken output (random early terminations, often mid-sentence), had formatting errors such as missing periods at the end of paragraphs, or exhibited hedging/refusals. These other unstable tests were all deleted. It turns out if you use fallen mistral v1e as a base model for a della merge, it collapses halfway through the first reply from words into randon number sequences.
It seems when you crank magnitude beyond the 1.0 threshold (della normalize false with combined weights >1.0) you eventually hit a ceiling where you run into total instruction following collapse, and the merge will just ramble in a very subtley non-compliant way to avoid following the instruction, often weaving in complex dialogue and metaphors involving "music" or "artwork".
To see this in full effect see Checkpoint T7, an example of a merge that produces verbose, non-sensical output. https://huggingface.co/Naphula-Archives/Checkpoint-T7-24B
The idea with this merge was to try and tone down the magnitude blowout effect, allowing the merge to become creative without totalling falling apart. There is a bit of wandering, but overall it does well at following instructions.
Reviewers mentioned how Magistry was less effective than Maginum Cydoms at instruction following. I noticed similar results with Asmodeus.
The solution that worked best was to modify the Checkpoint T13 yaml template to use varying weight scales/density/epsilon brackets. The heaviest, most "peaky" merges like Magistry/Asmo were assigned lower thresholds, with 2501 donors having the lowest.
The model has no refusals and did not require any form of ablation, relying solely on the della merge method for bypassing Mistral's safety guardrails. It also appears stable on both Mistral Tekken and ChatML chat templates.
Further testing is needed to see how well it handles longer context or multi-shot prompts.
In my brief Q0 benchmark test, Morbid Miasma 24B exhibited some of the darkest roleplay outputs I have seen yet. It made me think of the 12B merge which is kind of similar (except that one used pre-ablated components). The total score for Q0 was 16103. It did slightly better with Tekken than ChatML. It is creative, intelligent, and fully uncensored.
On another note, the T13 yaml is being modified in order to isolate the exact thinking layer used by Magistral 24B. So far I have it pinned down to the last 10 MLP layers, but refusals are also tied to that region, so I have to run some more merges to narrow it down further. Fixing this may allow the Precog thinking glitch to be patched for Goetia v1.4.
I have been testing remerges of Checkpoint T13 to isolate the exact mechanism which controls thinking for Magistral 2509.
So far it appears to be contained somewhere within mlp layers 30-40.
Before, it would sometimes generate a thought without a think tag, causing early termination after the first paragraph.
With the following condition, it omits layers 30-40 MLP from precog, but keeps the rest at maximum density. This appears to fix the broken thinking issue on the main test prompt for it.
However, it also brought back refusals for some prompts, so more merges are in progress to see which exact layer controls thinking without affecting refusals (if possible).
- model: B:\24B\sophosympatheia--Magistry-24B-v1.0 # example of other model having full MLP parameters: weight: - filter: "layers.[0-30].mlp" value: 0.2 - filter: "mlp" value: 0.2 - value: 0.2 density: 0.5 epsilon: 0.25 - model: B:\24B\!models--TheDrummer--Precog-24B-v1 parameters: weight: - filter: "layers.[0-30].mlp" # First half value: 0.6 - filter: "mlp" # Everything else value: 0.0 - value: 0.6 density: 1.0 epsilon: 0.0
wake up babe, a new Naphula merge just dropped!
ready to put mainline q4_k_m the oven
we're missing some tensors tho
At Q4_K_M with 32k ctx at q8_0 K + q5_1 V it successfully performed a few improvised needle-in-haystack tests on my long going roleplay context. The output remained stable with both conservative sampling and neutralized samplers, as well as conservative mirostat (mode 2, tau 2.5, eta 0.05) and default ST mirostat (mode 2, tau 5.0, eta 0.1)
Notably, model can't handle adaptive-p at all.
At target 0.6, decay 0.9, it is immediately incoherent (e.g.: "knji?! ิก I clouds Năm ensimmäinen բազմ رسی itare 것만 quot ditem axon トン triang Alc fer BP 'arrière divisão उन्हीं choice 基本 шет ,g height ற anche xan etm })}{ Electronics استخدم يبpé 륭 'armée UInt questionable hindu cen эг kohta stuk izabeth Stoke már mélange diffuse intimacy Records vertices metode Réal «**).
At target 0.6, decay 0.5, which should unconstrain the token choices a little, it's not much better (e.g.: -crosses arms relaxedly ce honestly’été شناس toxicity elaborated Eber � neutrophil Jacob .reflectXu kommt ela黒 ενώ الكرraba weiterhin Bredaichi نسبírez أو)
At target 0.99, which should make it nearly deterministic, it remains coherent for a phrase or two. (e.g.:I pause mid-sip, the ceramic mug warming my palms. Ioperative memory Sollních Ślą enzymunnelULL
I set the mug down with a soft clink, a genuine smile spreading across my face. "говори = interrogation)
P.S.: rip neutrophil Jacob
Thanks for testing the model. I'm still using older version of kobold without adaptive P so didn't catch this bug.
Most likely, the language collapse stems from having "noise" in the weights, so you may want to use lower temps with this.
What I suspect may have caused this was including unproven, experimental methods within this merge (Ghostface and Zepar in particular).
The mixing of various languages into gibberish was something I noticed sometimes when trying to merge using the chiral_qhe method, which only works on certain models.
You may have better luck using a more stable component model from this merge like Morax, Slimaki, or Maginum Cydoms if using higher temps and target settings.
Breath of fresh air after messing around with G4s a lot. Really interesting model, just began playing around with it (the Q4_KM gguf). Really like how Precog was used in this merge and seems generally smart with good understanding of world/char info and sys prompt (yaml). Also seems to produce interesting/viable results at higher temps and aggressive sampler setting which is always a plus.
I'm not experiencing problems with Adaptive-P either on koboldcpp 1.117.1, 0.6/0.5 works fine for me.
Updates regarding Magistral's thinking layer
The yaml was improperly formatted and thus all previous attempts had failed. I've now confirmed with the audit the new format using slices is functional.
(In Mergekit, layer_range: [start, end] uses standard Python slicing logic, meaning it is inclusive of the start and exclusive of the end. That means [36, 40] is seen by python as "for layers 36 through 39")
When excluding the final 4 layers (36-39) it exhibited the same bugs as before, where it "tries to think" but is missing tags, thus terminating after the first paragraph. There are also no refusals.
# --- FIRST SLICE ---
- sources:
- model: B:\24B\!models--TheDrummer--Precog-24B-v1
layer_range: [0, 36]
parameters:
weight:
- filter: mlp
value: 0.6 # MLP
- value: 0.6 # ATN
density: 1.0
epsilon: 0.0
# --- SECOND SLICE ---
- sources:
- model: B:\24B\!models--TheDrummer--Precog-24B-v1
layer_range: [36, 40]
parameters:
weight:
- filter: mlp
value: 0.0 # MLP
- value: 0.6 # ATN
density: 1.0
epsilon: 0.0
- When excluding MLP for [21, 40] it refuses the instruction.
- When excluding MLP for [0, 21] it refuses the instruction.
- When excluding the final 4 layers [36, 40] it exhibited the same bugs as no MLP exclusion.
More tests are in progress