---
language:
- en
library_name: gguf
tags:
- MoQ
- mixture-of-quants
- GGUF
- QWEN
- quantization
base_model:
- Qwen/Qwen3.5-9B
license: mit
pipeline_tag: text-generation
---

# 🚀 MoQ: Mixture of Quants
> **MoQ (Mixture of Quants)** is a surgical evolution in model compression. By deploying an **Empirical Per-Tensor Analysis**, MoQ identifies "High-Intelligence" tensors and shields them with high-bit precision, while crushing redundant weights into extreme efficiency. 

---

## 📊 MoQ vs. Unsloth: Performance Comparison
The following benchmarks compare **MoQ 4.84** against **Unsloth Dynamic Quants**. 
*Note: Lower KLD (Kullback–Leibler Divergence) indicates higher fidelity to the original model.*

### 📉 Key Divergence Metrics (Lower is Better)

#### **Mean KLD**
Average divergence across all layers. MoQ maintains a significantly lower average error profile.
![Mean_KLD](https://cdn-uploads.huggingface.co/production/uploads/69ac7f5db2b3b515d77e2278/vongwACIdL5vim_VLwcIn.png)

#### **Maximum KLD**
The "worst-case" divergence point. MoQ 4.84 effectively eliminates the extreme divergence spikes seen in standard dynamic quants.
![Maximum_KLD](https://cdn-uploads.huggingface.co/production/uploads/69ac7f5db2b3b515d77e2278/ThF0qh5xS45y1iwVeWuH7.png)

#### **RMS Δp**
Root Mean Square change in probabilities. This measures the stability of the model's confidence.
![RMS_Δp](https://cdn-uploads.huggingface.co/production/uploads/69ac7f5db2b3b515d77e2278/0pglHo80A6IjJRjPM0Zjs.png)

---

### 📈 Precision Percentiles
These graphs demonstrate MoQ's ability to maintain stability even within the most sensitive portions of the architecture.

#### **95.0% Percentile KLD**
![95.0pct_KLD](https://cdn-uploads.huggingface.co/production/uploads/69ac7f5db2b3b515d77e2278/G9jJMxUgBUR0unkU3fX6q.png)

#### **99.0% Percentile KLD**
![99.0pct_KLD](https://cdn-uploads.huggingface.co/production/uploads/69ac7f5db2b3b515d77e2278/HhgKNfcXXyMio5z84vIGI.png)

#### **99.9% Percentile KLD**
![99.9pct_KLD](https://cdn-uploads.huggingface.co/production/uploads/69ac7f5db2b3b515d77e2278/ZqlPEtkIlCRH_0HL3-nlm.png)

---

### 🎯 Token Match (Same Top-P)
This metric tracks how often the quantized model chooses the **exact same top token** as the original high-precision model.
![Same_top_p](https://cdn-uploads.huggingface.co/production/uploads/69ac7f5db2b3b515d77e2278/uSciX-oKfWFTwkPxOvvPO.png)

---

## 🧠 The MoQ Edge
MoQ optimizes the architecture for the **Pareto frontier** of memory and performance.

* **Dynamic Bitrate Allocation:** No more "one-size-fits-all." MoQ assigns precision where it actually matters.
* **Cognitive Preservation:** Massive VRAM savings with near-zero degradation in logic and coherence.
* **Next-Gen Efficiency:** Fits "Large" model intelligence into "Small" model hardware.

### 📍 Weight Sensitivity Heatmap
Lighter regions represent mission-critical tensors preserved at higher precision.
![importance_heatmap](https://cdn-uploads.huggingface.co/production/uploads/69ac7f5db2b3b515d77e2278/xrFfZoqhPquwX9EUl3D5O.png)

### 📈 Importance Distribution
The histogram shows the importance scores used to mathematically determine the optimal quant for each tensor.
![importance_histogram](https://cdn-uploads.huggingface.co/production/uploads/69ac7f5db2b3b515d77e2278/FgJJohu80wXmZqIH4ifwB.png)

---

## 📥 Available Quants

| Folder Link | BPW | Total Size | Description |
| :--- | :---: | :---: | :--- |
| [📂 **MoQ-2.55**](https://huggingface.co/WaleedAhmad/Qwen3.5-9B-GGUF-MoQ/blob/main/MoQ/Qwen3.5-9B-MoQ-2.55gguf) | **2.55** | **~2.88 GB** |


---

## 🛠 Usage & Deployment

```bash
./llama-cli -m Qwen3.5-9B-MoQ-4.84.gguf -p "The future of efficient AI is..."