---
language:
- en
library_name: gguf
tags:
- MoQ
- mixture-of-quants
- GGUF
- QWEN
- quantization
base_model:
- Qwen/Qwen3.5-9B
license: mit
pipeline_tag: text-generation

---
I am working on a new method that models the relationship between quantization error and the tensor type being quantized. It will be completed by Wednesday, after which I will delete the previous models and upload the new ones. I have observed that using different quantization types for the same tensor type (e.g., FFN down) across layers (1–23) results in poor quality and wastes the bit budget.
# 🚀 MoQ: Mixture of Quants

>MoQ (Mixture of Quants) is a smart way to shrink AI models without losing their "brainpower." Unlike old methods that treat every part of the model the same, MoQ identifies the most important parts and keeps them high-quality, while heavily compressing the rest to save space.****Stop settling for uniform bitrates.** Standard quantization is a relic of the past, treating vital cognitive weights the same as redundant noise. **MoQ (Mixture of Quants)** is a surgical evolution in model compression. By deploying an **Empirical Per-Tensor Analysis**, MoQ identifies the "High-Intelligence" tensors that drive reasoning and shields them with high-bit precision, while crushing redundant weights into extreme efficiency. 
---
The result? A model that punches significantly above its weight class.


---

## 🧠 The MoQ Edge
MoQ optimizes the architecture for the **Pareto frontier** of memory and performance.

* **Dynamic Bitrate Allocation:** No more "one-size-fits-all." MoQ assigns precision where it actually matters.
* **Cognitive Preservation:** Massive VRAM savings with near-zero degradation in logic and coherence.
* **Next-Gen Efficiency:** Fits "Large" model intelligence into "Small" model hardware.

### 📍 Weight Sensitivity Heatmap
Lighter regions represent mission-critical tensors preserved at higher precision.
![importance_heatmap](https://cdn-uploads.huggingface.co/production/uploads/69ac7f5db2b3b515d77e2278/xrFfZoqhPquwX9EUl3D5O.png)

### 📈 Importance Distribution
The histogram shows the importance scores used to mathematically determine the optimal quant for each tensor.
![importance_histogram](https://cdn-uploads.huggingface.co/production/uploads/69ac7f5db2b3b515d77e2278/FgJJohu80wXmZqIH4ifwB.png)

##
Follow me on Linkedin 
linkedin.com/in/waleed-ahmad-8a3166403
If MoQ does not perform well, email me :
waleedahmad.1a10@gmail.com

## 🛠 Usage & Deployment

```bash
./llama-cli -m Qwen3.5-9B-MoQ-4.85.gguf -p "The future of efficient AI is..."