--- license: mit pipeline_tag: text-generation tags: - glm - unsloth - MOE - pruning - compression - RSIT - Infiniteons base_model: - cerebras/GLM-4.7-Flash-REAP-23B-A3B library_name: transformers --- Use this model for **FREE** in your **Local VS Code**, even without a GPU card, via Google Colab (colab as endpoint): https://colab.research.google.com/gist/masterofrisk/d98c35f9f35a8350b11da6a94e29fa6e/free-glm-4-7-flash-colab_tnt_bridge.ipynb **Quantized from `unsloth/GLM-4.7-Flash-REAP-23B-A3B-GGUF` (BF16) using the Infiniteon Algebra associator to guide mixed-precision allocation: 20% of the weights (the shared expert and attention tensors) are quantized to 4-bit (Q4_K), and the remaining 80% to 3-bit (Q3_K). This achieves Q4-level generation quality at near-Q3 size, fitting a 23B-parameter MoE model inside 12 GB VRAM with a 150K context window.** **Why this works:** The associator of the Infiniteon algebra identifies which weight tensors carry structural "gap-crossing" information โ€” the tensors that connect diverse token neighborhoods. On MoE models, these are exactly the shared expert and attention tensors, which are used on every token. By allocating 4-bit precision to these structurally critical tensors and 3-bit to the rest, the model preserves the directional diversity needed for coherent generation while keeping the overall size near Q3. This is the first Q3Q4-based quantization of this model that generates without looping or repetitive collapse. **Research and credits:** This research was conducted by TNT.Chat (https://ai.tnt.chat) as part of our LOCAL AI effort to bring larger models onto consumer hardware. The target is a capable coder model running on 12 GB VRAM for VS Code usage with a 150,000-token context window. Additional models targeting RTX 5090 and RTX 6000 PRO (full GLM-4.7 218B REAP) are in progress. **Theory references:** Pinto Martins, C. F. (2026). *Ramsey Statistics And Infiniteons Theory, Volume I.* Zenodo. DOI: https://doi.org/10.5281/zenodo.19329589 Pinto Martins, C. F. (2026). *Ramsey Statistics And Infiniteons Theory, Volume II.* Zenodo. DOI: https://doi.org/10.5281/zenodo.19330373 --- **Suggested config for llama.cpp with a 150K context window fitting inside 12 GB VRAM (e.g., RTX 4070 Super):** ```batch set BINARY=llama-server set MODEL=GLM47-MIX20_Q3Q4.gguf set NP=1 "%BINARY%" ^ --model "%MODEL%" ^ --host 127.0.0.1 --port 4777 ^ --n-gpu-layers -1 --ctx-size 150000 ^ --parallel %NP%" --kv-unified ^ --flash-attn on ^ --cache-type-k q4_0 --cache-type-v q4_0 ^ -b 8192 -ub 2048 ^ --fit off --no-mmap --mlock ^ --reasoning on ^ --reasoning-format deepseek ^ --reasoning-budget 4096 ^ --reasoning-budget-message "Conclude the reasoning and provide the final answer." ^ --chat-template-kwargs '{"enable_thinking":true}' ^ --no-warmup ^ --jinja ``` --- This model was obtained by uniformly pruning 25% of experts in GLM-4.7-Flash using the REAP method. readme: > https://huggingface.co/cerebras/GLM-4.7-Flash-REAP-23B-A3B/main/README.md license_link: https://huggingface.co/zai-org/GLM-4.7-Flash/blob/main/LICENSE pipeline_tag: text-generation --- > [!NOTE] > Includes Unsloth **chat template fixes**!
For `llama.cpp`, use `--jinja` >

Unsloth Dynamic 2.0 achieves superior accuracy & outperforms other leading quants.

๐“Œณ REAP๐“Œณ the Experts: Why Pruning Prevails for One-Shot MoE Compression
REAP

# GLM-4.7-Flash-REAP-23B-A3B ## โœจ Highlights Introducing **GLM-4.7-Flash-REAP-23B-A3B**, a **memory-efficient compressed variant** of GLM-4.7-Flash that maintains near-identical performance while being **25% lighter**. This model was created using **REAP (Router-weighted Expert Activation Pruning)**, a novel expert pruning method that selectively removes redundant experts while preserving the router's independent control over remaining experts. Key features include: - **Near-Lossless Performance**: Maintains almost identical accuracy on code generation, agentic coding, and function calling tasks compared to the full 30B model - **25% Memory Reduction**: Compressed from 30B to 23B parameters, significantly lowering deployment costs and memory requirements - **Preserved Capabilities**: Retains all core functionalities including code generation, agentic workflows, repository-scale understanding, and function calling - **Drop-in Compatibility**: Works with vanilla vLLM - no source modifications or custom patches required - **Optimized for Real-World Use**: Particularly effective for resource-constrained environments, local deployments, and academic research --- ## ๐Ÿ“‹ Model Overview **GLM-4.7-Flash-REAP-23B-A3B** has the following specifications: - **Base Model**: GLM-4.7-Flash - **Compression Method**: REAP (Router-weighted Expert Activation Pruning) - **Compression Ratio**: 25% expert pruning - **Type**: Sparse Mixture-of-Experts (SMoE) Causal Language Model - **Number of Parameters**: 23B total, 3B activated per token - **Number of Layers**: 47 - **Number of Attention Heads**: 20 for QKV - **Number of Experts**: 48 (uniformly pruned from 64) - **Number of Activated Experts**: 4 per token - **Context Length**: 202,752 tokens - **License**: MIT --- ## ๐Ÿ“Š Evaluations
Benchmark GLM-4.7-Flash GLM-4.7-Flash-REAP-23B-A3B
Compression โ€” 25%
Coding
HumanEval 94.5 95.1
HumanEval+ 89.0 89.0
๐ŸŸฉ *This checkpoint maintains almost identical performance while being 25% lighter.* For more details on the evaluation setup, refer to the [REAP arXiv preprint](https://arxiv.org/abs/2510.13999). --- ## ๐Ÿš€ Deployment You can deploy the model directly using the **latest vLLM** (that supports GLM4.7-Flash), no source modifications or custom patches required. ```bash vllm serve cerebras/GLM-4.7-Flash-REAP-23B-A3B \ --tensor-parallel-size 4 \ --reasoning-parser glm45 \ --tool-call-parser glm47 \ --enable-auto-tool-choice ``` If you encounter insufficient memory when running this model, you might need to set a lower value for `--max-num-seqs` flag (e.g. set to 64). ## ๐Ÿงฉ Model Creation This checkpoint was created by applying the **REAP (Router-weighted Expert Activation Pruning)** method uniformly across all Mixture-of-Experts (MoE) blocks of **GLM-4.7**, with a **25% pruning rate**. ### How REAP Works REAP selects experts to prune based on a novel **saliency criterion** that considers both: - **Router gate values**: How frequently and strongly the router activates each expert - **Expert activation norms**: The magnitude of each expert's output contributions This dual consideration ensures that experts contributing minimally to the layer's output are pruned, while preserving those that play critical roles in the model's computations. ### Key Advantages - **One-Shot Compression**: No fine-tuning required after pruning - the model is immediately ready for deployment - **Preserved Router Control**: Unlike expert merging methods, REAP maintains the router's independent, input-dependent control over remaining experts, avoiding "functional subspace collapse" - **Generative Task Superiority**: REAP significantly outperforms expert merging approaches on generative benchmarks (code generation, creative writing, mathematical reasoning) while maintaining competitive performance on discriminative tasks ### Calibration The model was calibrated using a diverse mixture of domain-specific datasets including: - Code generation samples ([evol-codealpaca](https://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1)) - Function calling examples ([xlam-function-calling](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k)) - Agentic multi-turn trajectories ([SWE-smith-trajectories](https://huggingface.co/datasets/SWE-bench/SWE-smith-trajectories)) ๐Ÿ“š For more details, refer to the following resources: - [๐Ÿงพ arXiv Preprint](https://arxiv.org/abs/2510.13999) - [๐Ÿงพ REAP Blog](https://www.cerebras.ai/blog/reap) - [๐Ÿ’ป REAP Codebase (GitHub)](https://github.com/CerebrasResearch/reap) --- ## โš–๏ธ License This model is derived from **[`zai-org/GLM-4.7-Flash`](https://huggingface.co/zai-org/GLM-4.7-Flash)** and distributed under the **MIT license**. --- ## ๐Ÿงพ Citation If you use this checkpoint, please cite the REAP paper: ```bibtex @article{lasby-reap, title={REAP the Experts: Why Pruning Prevails for One-Shot MoE compression}, author={Lasby, Mike and Lazarevich, Ivan and Sinnadurai, Nish and Lie, Sean and Ioannou, Yani and Thangarasa, Vithursan}, journal={arXiv preprint arXiv:2510.13999}, year={2025} } ```