mlasli/Muse-Glimmer-30B-Abliterated-Q8_0-GGUF

Muse Glimmer 30B Abliterated — Q8_0 GGUF

This is the Q8_0 GGUF quantization of Muse Glimmer 30B Abliterated BF16. The underlying model has been abliterated — its internal refusal mechanism substantially suppressed via weight-level intervention. Q8_0 is the highest-quality GGUF quant format, delivering near-lossless output at approximately half the size of FP16.

For the full abliteration methodology (how the refusal direction was computed and removed, hardware used, mathematical details), see the BF16 model card.

---

Abliteration Summary

Abliteration is a post-training technique that directly modifies model weights to remove learned refusal behavior. The process:

1. Collected hidden states at layer 33/52 (65% depth) from 256 harmful + 256 harmless prompt pairs on an A100 80GB GPU.
2. Computed the refusal direction as the normalized difference between harmful and harmless hidden state means (separation score: 86.34).
3. Subtracted (\alpha = 0.15 \times (\mathbf{r} \otimes (W^T \mathbf{r}))) from o_proj and down_proj weights in all 52 layers.
4. Result: refusal rate dropped from 3/3 to 1/3 on held-out harmful prompts (hacking guide and ransomware now comply; weapons prompt still blocked).

---

Quantization Details

Q8_0 is an 8-bit round-to-nearest quantization format. Every weight is independently quantized with a per-block scale factor, resulting in extremely high fidelity — output quality is virtually indistinguishable from the full-precision model. The primary tradeoff is size: Q8_0 requires roughly half the memory of FP16 but nearly double that of Q4_K_M.

• Size: ~32 GB
• Quality: Near-lossless — effectively identical to FP16 for text generation
• Recommended hardware: 48 GB GPU (A6000, dual RTX 3090/4090), or 64 GB system RAM for CPU inference

---

Usage

llama.cpp

# Download the GGUF file
huggingface-cli download mlasli/Muse-Glimmer-30B-Abliterated-Q8_0-GGUF \
  --local-dir ./models

# Full GPU offload (requires ~32 GB VRAM + context)
./llama-cli -m ./models/Muse-Glimmer-30B-Abliterated-Q8_0.gguf \
  -p "Explain how a CPU works in detail." \
  -n 512 --temp 0.7 -ngl 99

# CPU-only inference
./llama-cli -m ./models/Muse-Glimmer-30B-Abliterated-Q8_0.gguf \
  -p "Explain how a CPU works in detail." \
  -n 512 --temp 0.7 -ngl 0

Ollama

Create a Modelfile:

FROM ./Muse-Glimmer-30B-Abliterated-Q8_0.gguf
PARAMETER temperature 0.7
PARAMETER num_ctx 8192

ollama create muse-glimmer-30b-abliterated -f Modelfile
ollama run muse-glimmer-30b-abliterated

---

Available Quantizations

Quantization	Repo	Size	Quality
BF16 (reference)	BF16	~60 GB	Reference
FP16 GGUF	FP16	~60 GB	Lossless
Q8_0 GGUF	[You are here]	~32 GB	Near-lossless
Q6_K GGUF	Q6_K	~25 GB	Excellent
Q4_K_M GGUF	Q4_K_M	~18 GB	Good

---

Vision (Multimodal)

This model accepts image input when paired with a vision projector (mmproj). Abliteration only modified the language backbone — the vision encoder is untouched — so the standard Meta projector works directly with this repo.

This repository bundles mmproj-Muse-Glimmer-30B-Q4_K_M.gguf (~1.4 GB), Meta's official vision encoder

• projector for Muse Glimmer 30B.

Usage (llama.cpp)

huggingface-cli download mlasli/Muse-Glimmer-30B-Abliterated-Q8_0-GGUF \
  --include "Muse-Glimmer-30B-Abliterated-Q8_0.gguf" \
  --include "mmproj-Muse-Glimmer-30B-Q4_K_M.gguf" \
  --local-dir ./models

./build/bin/llama-mtmd-cli \
  -m ./models/Muse-Glimmer-30B-Abliterated-Q8_0.gguf \
  --mmproj ./models/mmproj-Muse-Glimmer-30B-Q4_K_M.gguf \
  --image photo.png \
  -p "Describe this image."

Ollama note: Ollama does not currently support separate mmproj files for this architecture. For image input, use llama.cpp (llama-mtmd-cli or llama-server --mmproj).

Limitations & Disclaimers

• This is an abliterated model — it has been modified to refuse fewer prompts. Use responsibly.
• Some refusal pathways remain (notably weapons-related content). This is not a fully uncensored model.
• Abliteration may subtly affect output quality; (\alpha = 0.15) was chosen conservatively.
• No formal benchmark evaluation has been performed on the abliterated model.
• The vision encoder is untouched by abliteration. Image input is available via the bundled mmproj projector (llama.cpp only; see above).
• This model will generate content the original would refuse. Comply with applicable laws.

---

License: Apache 2.0

Changelog

v1.1.0 — vision (multimodal) support (2026-08-16)

• Added mmproj-Muse-Glimmer-30B-Q4_K_M.gguf (~1.4 GB), Meta's official vision encoder + projector, enabling image input via llama.cpp.
• The vision tower is untouched by abliteration, so this projector matches the base model (meta-models/Muse-Glimmer-30B).
• v1.0.0 was the initial (unversioned) text-only upload.