---
license: apache-2.0
base_model: Qwen/Qwen3.6-35B-A3B
pipeline_tag: text-generation
library_name: gguf
tags:
  - gguf
  - imatrix
  - llama.cpp
  - quantized
quantized_by: Krasnopjorovs
---

# Qwen3.6-35B-A3B — imatrix GGUF quantizations

GGUF imatrix builds of [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B).

Quantized with [llama.cpp](https://github.com/ggerganov/llama.cpp) `usage: /home/artjoms/llama.cpp/build/bin/llama-quantize [--help] [--allow-requan` using importance-matrix calibration on a public multilingual + code + math corpus.

## Prompt format

```
<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
```

## Available quants

| Filename | Quant | Size | Notes |
|---|---|---|---|
| [Qwen3.6-35B-A3B-Q8_0.gguf](https://huggingface.co/Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF/blob/main/Qwen3.6-35B-A3B-Q8_0.gguf) | `Q8_0` | 34.37 GB | Practically lossless. Closest to source with significant size cut. |
| [Qwen3.6-35B-A3B-Q6_K_L.gguf](https://huggingface.co/Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF/blob/main/Qwen3.6-35B-A3B-Q6_K_L.gguf) | `Q6_K` | 26.79 GB | Q6_K with Q8_0 embed/output tensors. Near-lossless top tier. |
| [Qwen3.6-35B-A3B-Q6_K.gguf](https://huggingface.co/Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF/blob/main/Qwen3.6-35B-A3B-Q6_K.gguf) | `Q6_K` | 26.56 GB | Near-lossless quality. Recommended for highest practical fidelity. |
| [Qwen3.6-35B-A3B-Q5_K_L.gguf](https://huggingface.co/Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF/blob/main/Qwen3.6-35B-A3B-Q5_K_L.gguf) | `Q5_K_M` | 23.32 GB | Q5_K_M with Q8_0 embed/output. High quality with small overhead. |
| [Qwen3.6-35B-A3B-Q5_K_M.gguf](https://huggingface.co/Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF/blob/main/Qwen3.6-35B-A3B-Q5_K_M.gguf) | `Q5_K_M` | 23.03 GB | High quality, balanced size. Recommended general-purpose. |
| [Qwen3.6-35B-A3B-Q5_K_S.gguf](https://huggingface.co/Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF/blob/main/Qwen3.6-35B-A3B-Q5_K_S.gguf) | `Q5_K_S` | 22.33 GB | Slightly smaller than Q5_K_M with similar quality. |
| [Qwen3.6-35B-A3B-Q4_K_L.gguf](https://huggingface.co/Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF/blob/main/Qwen3.6-35B-A3B-Q4_K_L.gguf) | `Q4_K_M` | 20.06 GB | Q4_K_M with Q8_0 embed/output. Sweet spot of quality and size. |
| [Qwen3.6-35B-A3B-Q4_K_M.gguf](https://huggingface.co/Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF/blob/main/Qwen3.6-35B-A3B-Q4_K_M.gguf) | `Q4_K_M` | 19.71 GB | Best size/quality tradeoff. Recommended default. |
| [Qwen3.6-35B-A3B-Q4_K_S.gguf](https://huggingface.co/Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF/blob/main/Qwen3.6-35B-A3B-Q4_K_S.gguf) | `Q4_K_S` | 18.52 GB | Compact with minor quality loss versus Q4_K_M. |
| [Qwen3.6-35B-A3B-IQ4_NL.gguf](https://huggingface.co/Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF/blob/main/Qwen3.6-35B-A3B-IQ4_NL.gguf) | `IQ4_NL` | 18.42 GB | Slightly larger than IQ4_XS. Online repacking for ARM CPU inference. |
| [Qwen3.6-35B-A3B-IQ4_XS.gguf](https://huggingface.co/Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF/blob/main/Qwen3.6-35B-A3B-IQ4_XS.gguf) | `IQ4_XS` | 17.44 GB | Most efficient sub-Q4. Smaller than Q4_K_S with comparable quality. |

## Calibration

Imatrix generated from [reapmix (community calibration mix)](https://huggingface.co/datasets/eaddario/imatrix-calibration) — ~400K tokens — multilingual + code + math. This is the same class of public calibration data used by other community GGUF publishers; no claim of unique calibration is made for this release.

`*_L` and `*_XL` variants override the output tensor and/or token embedding to Q8_0 (versus the base type), at small extra disk for typically improved output stability at low bit-rates.

## Download

Single file:

```bash
hf download Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF --include "Qwen3.6-35B-A3B-Q4_K_M.gguf" --local-dir .
```

Whole repo:

```bash
hf download Krasnopjorovs/Qwen3.6-35B-A3B-Imatrix-GGUF --local-dir ./Qwen3.6-35B-A3B-gguf
```

## Run

```bash
./llama-server -m Qwen3.6-35B-A3B-Q4_K_M.gguf -c 32768 -ngl 99 --host 0.0.0.0 --port 8080
```

## Picking a quant

- **Q8_0 / Q6_K_L** — RAM headroom, want ceiling quality
- **Q5_K_M / Q4_K_L** — workstation default, very small quality loss
- **Q4_K_M** — best general size/quality tradeoff, the default choice
- **IQ4_XS** — smallest with serious quality, for tight RAM
- **IQ3_M / Q3_K_M** — when 8GB-class VRAM is the budget
- **IQ2_M and below** — emergency only, quality degrades visibly

## Build info

- llama.cpp release: `usage: /home/artjoms/llama.cpp/build/bin/llama-quantize [--help] [--allow-requan`
- Host: iron-z-01 (x86_64)
- Generated: 2026-06-10T22:32:10

## Credits

- Original model by [Qwen](https://huggingface.co/Qwen)
- Calibration: [reapmix (community calibration mix)](https://huggingface.co/datasets/eaddario/imatrix-calibration)
- llama.cpp by ggerganov and contributors