---
base_model:
- Qwen/Qwen3.6-35B-A3B
library_name: gguf
license: apache-2.0
tags:
- qwen
- qwen3.6
- moe
- gguf
- iq4_xs
- imatrix
- text-generation
pipeline_tag: text-generation
---

# Qwen 3.6 35B A3B - GGUF (IQ4_XS) with Custom Imatrix

## 🚀 Model Overview
This repository contains a highly optimized, custom-quantized GGUF version of **Qwen 3.6 35B A3B**. 
It leverages the Mixture-of-Experts (MoE) architecture, possessing 35 Billion total parameters but activating only ~3 Billion parameters per token during inference. This provides flagship-level intelligence (advanced logic, coding, multilingual RAG) at unprecedented speeds.

## 🧠 Custom Quantization (The "Reapmix" Imatrix)
Unlike standard uniform quantizations that often degrade a model's reasoning capabilities, this specific build was compressed using a **Custom Importance Matrix (`.imatrix`)**.
- **Calibration Dataset:** Computed over 1.1 million strictly selected tokens (`reapmix_imatrix.txt`).
- **Format:** `IQ4_XS` (i-quants, Extra Small).
- **Bit-per-weight (BPW):** ~4.32.
- **Result:** The model size was dramatically reduced from ~66.1 GB to just **17.8 GB**, preserving near 100% of its deductive reasoning, JSON-formatting discipline, and constraint satisfaction abilities.

## 💻 Hardware Requirements
This build is designed to maximize VRAM efficiency, allowing a 35B model to fit comfortably on consumer and workstation GPUs while leaving massive headroom for the context window.
- **File Size:** ~17.8 GB.
- **Minimum VRAM:** 24 GB (e.g., RTX 3090, 4090, A5000, RTX 5000) for full GPU offload with 8k-16k context.

## 🛠️ How to Run

### 1. Using `llama.cpp` (Web Server Mode)
The most efficient way to run this model is via the `llama-server` binary with maximum GPU offload.

```bash
./llama-server -m qwen3.6-35B-A3B-IQ4_XS.gguf -c 32768 -ngl 99 --host 0.0.0.0 --port 8080

🎯 Use Cases Tested
- This specific quantization has been heavily verified against:
- Cross-Language RAG: Seamlessly bridging English data-center infrastructure rules with Russian situational queries.
- Algorithmic Coding: Generating O(N) complexity Python scripts without regex, strictly following constraint rules.
- Strict Formatting: Outputting pure, valid JSON objects without markdown wrappers or conversational filler.