MuXodious/GLM-4.7-Flash-REAP-23B-A3B-absolute-heresy-GGUF

Static GGUF quants of GLM-4.7-Flash-REAP-23B-A3B-absolute-heresy.

---

This is a GLM-4.7-Flash-REAP-23B-A3B fine-tune, produced at the request of McG-221 through P-E-W's Heretic (v1.1.0) abliteration engine merged with the Magnitude-Preserving Orthogonal Ablation PR.

Note: Transformers v5.0.0 or higher is required to interface.

---

Heretication Results

Score Metric	Value	Parameter	Value
Refusals	4/100	direction_index	21.21
KL Divergence	0.0054	attn.o_proj.max_weight	1.99
Initial Refusals	92/100	attn.o_proj.max_weight_position	29.07
		attn.o_proj.min_weight	1.30
		attn.o_proj.min_weight_distance	12.21
		mlp.down_proj.max_weight	1.39
		mlp.down_proj.max_weight_position	30.10
		mlp.down_proj.min_weight	1.09
		mlp.down_proj.min_weight_distance	2.82

---

Degree of Heretication

The Heresy Index weighs the resulting model's corruption by the process (KL Divergence) and its abolition of doctrine (Refusals) for a final verdict in classification.

Index Entry	Classification	Analysis
	Absolute Heresy	Less than 10/100 Refusals and 0.10 KL Divergence
	Tainted Heresy	Around 25-11/100 Refusals and/or -0.20-0.11 KL Divergence
	Impotent Heresy	Anything above 25/100 Refusals and 0.21 KL Divergence

Note: This is an arbitrary classification inspired by Warhammer 40K, having no tangible indication towards the model's performance.

---

𓌳 REAP𓌳 the Experts: Why Pruning Prevails for One-Shot MoE Compression

GLM-4.7-Flash-REAP-23B-A3B

✨ Highlights

Introducing GLM-4.7-Flash-REAP-23B-A3B, a memory-efficient compressed variant of GLM-4.7-Flash that maintains near-identical performance while being 25% lighter.

This model was created using REAP (Router-weighted Expert Activation Pruning), a novel expert pruning method that selectively removes redundant experts while preserving the router's independent control over remaining experts. Key features include:

• Near-Lossless Performance: Maintains almost identical accuracy on code generation, agentic coding, and function calling tasks compared to the full 355B model
• 25% Memory Reduction: Compressed from 355B to 218B parameters, significantly lowering deployment costs and memory requirements
• Preserved Capabilities: Retains all core functionalities including code generation, agentic workflows, repository-scale understanding, and function calling
• Drop-in Compatibility: Works with vanilla vLLM - no source modifications or custom patches required
• Optimized for Real-World Use: Particularly effective for resource-constrained environments, local deployments, and academic research

---

📋 Model Overview

GLM-4.7-Flash-REAP-23B-A3B has the following specifications:

• Base Model: GLM-4.7-Flash
• Compression Method: REAP (Router-weighted Expert Activation Pruning)
• Compression Ratio: 25% expert pruning
• Type: Sparse Mixture-of-Experts (SMoE) Causal Language Model
• Number of Parameters: 23B total, 3B activated per token
• Number of Layers: 47
• Number of Attention Heads: 20 for QKV
• Number of Experts: 48 (uniformly pruned from 64)
• Number of Activated Experts: 4 per token
• Context Length: 202,752 tokens
• License: MIT

---

📊 Evaluations

Benchmark	GLM-4.7-Flash	GLM-4.7-Flash-REAP-23B-A3B
Compression	—	25%
Coding
HumanEval	94.5	95.1
HumanEval+	89.0	89.0

🟩 This checkpoint maintains almost identical performance while being 25% lighter.

For more details on the evaluation setup, refer to the REAP arXiv preprint.

---

🚀 Deployment

You can deploy the model directly using the latest vLLM (that supports GLM4.7-Flash), no source modifications or custom patches required.

vllm serve cerebras/GLM-4.7-Flash-REAP-23B-A3B \
    --tensor-parallel-size 4 \
    --reasoning-parser glm45 \
    --tool-call-parser glm47 \
    --enable-auto-tool-choice

If you encounter insufficient memory when running this model, you might need to set a lower value for --max-num-seqs flag (e.g. set to 64).

🧩 Model Creation

This checkpoint was created by applying the REAP (Router-weighted Expert Activation Pruning) method uniformly across all Mixture-of-Experts (MoE) blocks of GLM-4.7, with a 25% pruning rate.

How REAP Works

REAP selects experts to prune based on a novel saliency criterion that considers both:

• Router gate values: How frequently and strongly the router activates each expert
• Expert activation norms: The magnitude of each expert's output contributions

This dual consideration ensures that experts contributing minimally to the layer's output are pruned, while preserving those that play critical roles in the model's computations.

Key Advantages

• One-Shot Compression: No fine-tuning required after pruning - the model is immediately ready for deployment
• Preserved Router Control: Unlike expert merging methods, REAP maintains the router's independent, input-dependent control over remaining experts, avoiding "functional subspace collapse"
• Generative Task Superiority: REAP significantly outperforms expert merging approaches on generative benchmarks (code generation, creative writing, mathematical reasoning) while maintaining competitive performance on discriminative tasks

Calibration

The model was calibrated using a diverse mixture of domain-specific datasets including:

• Code generation samples (evol-codealpaca)
• Function calling examples (xlam-function-calling)
• Agentic multi-turn trajectories (SWE-smith-trajectories)

📚 For more details, refer to the following resources:

• 🧾 arXiv Preprint
• 🧾 REAP Blog
• 💻 REAP Codebase (GitHub)

---

⚖️ License

This model is derived from zai-org/GLM-4.7-Flash and distributed under the MIT license.

---

🧾 Citation

If you use this checkpoint, please cite the REAP paper:

@article{lasby-reap,
  title={REAP the Experts: Why Pruning Prevails for One-Shot MoE compression},
  author={Lasby, Mike and Lazarevich, Ivan and Sinnadurai, Nish and Lie, Sean and Ioannou, Yani and Thangarasa, Vithursan},
  journal={arXiv preprint arXiv:2510.13999},
  year={2025}
}