LuffyTheFox

/

Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V10-GGUF

like 561

I really appreciate your attitude towards model tuning keep on,sir

What are the changes you made in the V8 (Hermes3.6-35B-A3B-Uncensored-Genesis-V8-APEX-Compact.gguf uploaded 1 hour ago) ? It would be cool to have a change list somewhere.

What are the changes you made in the V8 (Hermes3.6-35B-A3B-Uncensored-Genesis-V8-APEX-Compact.gguf uploaded 1 hour ago) ? It would be cool to have a change list somewhere.

Here info: https://huggingface.co/LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V7-GGUF/discussions/39

Independent verification of your conv1d diagnosis — full benchmark results

Thank you for this write-up — it prompted us to run an independent, controlled verification, and your diagnosis holds up. We are the hermes-nova team (org redashes), working with official Qwen/Qwen3.8-27B BF16 weights.

1. Diagnosis confirmed by independent measurement

We measured linear_attn.conv1d.weight std across the 48 GatedDeltaNet layers on the official BF16 weights, and flagged the same 8 layers using a std/median > 1.6 rule:

Layer set and α values are identical — two independent implementations, convergent diagnosis. α ≈ 0.48–0.65, exactly your range.

2. What a strict per-layer α fix does — measured, not asserted

We applied your strict per-layer α (not median normalization), kept everything else bit-exact, and ran a full controlled battery on the exact same stack (vLLM 0.27.1, same harness, same hardware: single RTX PRO 6000).

9 of 11 metrics ≥ official. Generation quality and truthfulness improve substantially (TruthfulQA-gen +6~8pp, MT-Bench +0.42), and the "overthinking" pattern you documented is visibly reduced. The single notable cost: CMMLU −1.8pp (knowledge-heavy closed-book recall).

3. On "you cannot fix this by patching a few tensors"

Partial refutation, with nuance. A small tensor patch does move the model meaningfully — our data shows it is not a free lunch, but it is also not a write-off:

• Generative metrics moved large (TruthfulQA-gen +6~8pp) — the drift was real and the fix effective.
• But CMMLU −1.8pp confirms your broader point that the repair trades knowledge capacity for generation quality; the pathway damage is not simply reversible.

So: your diagnosis is confirmed, your pessimism is partially justified, and the measured verdict is "worth it for generative use, not for knowledge-max use".

4. Release (public, weights + full eval + method)

redashes/Qwen3.8-27B-BF16-SSMFIX — now public on Hugging Face, with bilingual README, full eval tables, fix report, and method credit to you and FGDumitru (qwen-ssm-repair, who independently re-implemented this and got overlapping α 0.61–0.70):

https://huggingface.co/redashes/Qwen3.8-27B-BF16-SSMFIX

Thanks again for the original investigation — it was the right lead.

Independent verification of your conv1d diagnosis — full benchmark results

Thank you for this write-up — it prompted us to run an independent, controlled verification, and your diagnosis holds up. We are the hermes-nova team (org redashes), working with official Qwen/Qwen3.8-27B BF16 weights.

1. Diagnosis confirmed by independent measurement

We measured linear_attn.conv1d.weight std across the 48 GatedDeltaNet layers on the official BF16 weights, and flagged the same 8 layers using a std/median > 1.6 rule:

Layer	our α	your α	match
52	0.59005	0.59005	✓
53	0.55484	0.55484	✓
56	0.54486	0.54486	✓
57	0.53574	0.53574	✓
58	0.60972	0.60972	✓
60	0.48136	0.48136	✓
61	0.65327	0.65327	✓
62	0.61856	0.61856	✓

Layer set and α values are identical — two independent implementations, convergent diagnosis. α ≈ 0.48–0.65, exactly your range.

2. What a strict per-layer α fix does — measured, not asserted

We applied your strict per-layer α (not median normalization), kept everything else bit-exact, and ran a full controlled battery on the exact same stack (vLLM 0.27.1, same harness, same hardware: single RTX PRO 6000).

Metric	official BF16	+SSMFIX (per-layer α)	Δ
MT-Bench avg (DeepSeek-V4-Flash judge)	7.05	7.47	+0.42
IFEval inst-level strict	0.6247	0.6343	+0.96pp
GSM8K strict (5-shot)	0.9606	0.9644	+0.38pp
TruthfulQA gen rouge1	0.2840	0.3452	+6.1pp
TruthfulQA gen rouge2	0.1616	0.2460	+8.4pp
TruthfulQA gen rougeL	0.2803	0.3452	+6.5pp
TruthfulQA gen bleu	0.1775	0.2558	+7.8pp
TruthfulQA mc1 / mc2	0.3647/0.5418	0.3758/0.5513	+0.0111 / +0.0095
CMMLU (68 subtasks)	0.7179	0.6996	−1.8pp

9 of 11 metrics ≥ official. Generation quality and truthfulness improve substantially (TruthfulQA-gen +6~8pp, MT-Bench +0.42), and the "overthinking" pattern you documented is visibly reduced. The single notable cost: CMMLU −1.8pp (knowledge-heavy closed-book recall).

3. On "you cannot fix this by patching a few tensors"

Partial refutation, with nuance. A small tensor patch does move the model meaningfully — our data shows it is not a free lunch, but it is also not a write-off:

• Generative metrics moved large (TruthfulQA-gen +6~8pp) — the drift was real and the fix effective.
• But CMMLU −1.8pp confirms your broader point that the repair trades knowledge capacity for generation quality; the pathway damage is not simply reversible.

So: your diagnosis is confirmed, your pessimism is partially justified, and the measured verdict is "worth it for generative use, not for knowledge-max use".

4. Release (public, weights + full eval + method)

redashes/Qwen3.8-27B-BF16-SSMFIX — now public on Hugging Face, with bilingual README, full eval tables, fix report, and method credit to you and FGDumitru (qwen-ssm-repair, who independently re-implemented this and got overlapping α 0.61–0.70):

https://huggingface.co/redashes/Qwen3.8-27B-BF16-SSMFIX

Thanks again for the original investigation — it was the right lead.

Thank you very much for sharing testing results with SSM fix. This is the right path to take open source. We should not blindly use models shared by companies. A neural network is not a black box; it is a statistical machine governed by fundamental mathematical laws. Let's hope Alibaba Group will fix this issue in Qwen4.0 during training process.