hipfire-models/Qwen3.6-27B-Uncensored-mq4

Qwen3.6-27B-Uncensored — hipfire MQ4 (AWQ)

A 4-bit hipfire MQ4 quantization of an abliterated (uncensored) Qwen3.6-27B, built for Rust-native inference on AMD RDNA / CDNA GPUs.

• Base model: huihui-ai/Huihui-Qwen3.6-27B-abliterated (BF16 safetensors)
• Architecture: Qwen3.6-27B dense — 64 layers (48 Gated-DeltaNet linear-attention + 16 full gated-attention), hidden 5120, FFN 17408, vocab 248320
• Format: hipfire MQ4G256 (~4.4 bpw), single self-contained .mq4 file
• Text only: the base model's vision tower (mmproj) is not included

⚠️ This model ships AWQ scales

This is not a plain MQ4 quant. It carries AWQ (Activation-aware Weight Quantization) pre-scaling baked in:

• Each quantized linear weight has a companion <weight_name>.awq_scale.weight tensor (1-D F16, length = input dim K) embedded in the same .mq4 file.
• During quantization, weights were pre-scaled per input channel by s[j] = (RMS_act[j])^α, α = 0.5 (paper formula, geo-mean-normalized to 1).
• At inference the hipfire runtime divides the activation by awq_scale (x /= awq_scale before the rotation kernel), exactly compensating the pre-scale so the math is equivalent to the unscaled product — while the 4-bit grid is spent where activations are largest.

Compatibility: the hipfire runtime auto-detects and applies these scales by tensor name (see hipfire-runtime::hfq::load_awq_scale), not by file extension — so the plain .mq4 filename is correct and the AWQ behavior is preserved. A loader that ignores *.awq_scale.weight would read these weights as if un-prescaled and produce corrupt logits, so use a hipfire build that supports AWQ.

The lm_head is plain MQ4 with no AWQ scale (the safe default — avoids the mismatched-head logit-corruption failure mode).

Importance matrix (imatrix)

AWQ per-channel scales were derived from a hipfire-native imatrix generated with llama-imatrix over an agentic calibration corpus (hermes agent-reasoning traces, ChatML-flattened — not wikitext), 285 chunks × 2048 tokens.

Usage (hipfire)

hipfire serve --model qwen3.6-27b-uncensored.mq4
# or a one-shot eval / coherence probe:
coherence_probe --model qwen3.6-27b-uncensored.mq4 \
  --prompt-file prompt.txt --max-tokens 200 --temperature 0.0

Recommended KV mode: fwht3 (the asym3 default is lower quality).

Provenance / caveats

• Quantized on an MI300x (gfx942). Coherence-gate: 0 hard / 0 soft across all detectors (attractor / n-gram / loop-guard / special-leak / think-empty / eos).
• This is a different abliteration recipe than the HauhauCS "Balanced" GGUFs; it derives from the huihui-ai abliterated weights.
• Uncensored model: alignment guardrails have been ablated from the base weights. Use responsibly and in accordance with the Apache-2.0 license and the Qwen terms.