---
license: other
base_model: lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled
base_model_relation: quantized
quantized_by: blockblockblock
library_name: exllamav3
pipeline_tag: text-generation
tags:
  - exl3
  - exllamav3
  - quantized
  - mixture-of-experts
  - qwen
quantization_format: exl3
bits_per_weight: 4.5
---

<div align="center">

# Qwen3.6 · 35B-A3B · Claude 4.7 Opus Reasoning Distilled

<sub><code>EXL3</code> &nbsp;·&nbsp; <b>4.5&nbsp;bpw</b> &nbsp;·&nbsp; 21.6&nbsp;GB &nbsp;·&nbsp; Mixture&#8209;of&#8209;Experts &nbsp;·&nbsp; 48 layers &times; 256 experts</sub>

<br/>

[![format](https://img.shields.io/badge/format-EXL3-c63010?style=for-the-badge&labelColor=14120e)](https://github.com/turboderp-org/exllamav3)
[![bpw](https://img.shields.io/badge/bpw-4.5-6b8a76?style=for-the-badge&labelColor=14120e)](#quants)
[![size](https://img.shields.io/badge/size-21.6_GB-6b8a76?style=for-the-badge&labelColor=14120e)](#quants)
[![arch](https://img.shields.io/badge/arch-MoE_35B--A3B-c63010?style=for-the-badge&labelColor=14120e)](https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled)

[![base model](https://img.shields.io/badge/Base-lordx64%2FQwen3.6--35B--A3B--Claude--4.7--Opus--Reasoning--Distilled-2a2620?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled)
[![quantized by](https://img.shields.io/badge/Quantized_by-blockblockblock-2a2620?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/blockblockblock)
[![collection](https://img.shields.io/badge/All_bpws-Collection-c63010?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/collections/blockblockblock/qwen36-35b-a3b-claude-47-opus-reasoning-distilled-exl3-69ece89b196fa4ae78d37550)

</div>

---

> [!NOTE]
> An [ExLlamaV3](https://github.com/turboderp-org/exllamav3) build of [`lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled`](https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled) at **4.5 bits per weight** &mdash; the quality-leaning sweet spot: comfortable on a single 24&nbsp;GB consumer GPU, effectively indistinguishable from FP16 on most reasoning tasks. See [Quants](#quants) for sibling repos at other bit&#8209;widths or browse the [collection](https://huggingface.co/collections/blockblockblock/qwen36-35b-a3b-claude-47-opus-reasoning-distilled-exl3-69ece89b196fa4ae78d37550).

## Quants

<div align="center">

| BPW &nbsp; | &nbsp; Head bits &nbsp; | &nbsp; Calibration rows &nbsp; | &nbsp; Size &nbsp; | &nbsp; Status |
| :---: | :---: | :---: | ---: | :--- |
| 3.0 | 8 | 128 | 15.3&nbsp;GB | [link](https://huggingface.co/blockblockblock/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-exl3-3.0bpw) |
| 4.0 | 8 | 128 | 19.5&nbsp;GB | [link](https://huggingface.co/blockblockblock/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-exl3-4.0bpw) |
| **4.5** | 8 | 250 | **21.6&nbsp;GB** | <kbd>this repo</kbd> |
| 5.0 | 8 | 250 | 23.7&nbsp;GB | [link](https://huggingface.co/blockblockblock/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-exl3-5.0bpw) |
| 6.0 | 8 | 250 | 27.9&nbsp;GB | [link](https://huggingface.co/blockblockblock/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-exl3-6.0bpw) |

</div>

## Inference

<table>
  <thead>
    <tr>
      <th align="left" width="32%">Loader</th>
      <th align="left">Use it for</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><a href="https://github.com/theroyallab/tabbyAPI"><b>TabbyAPI</b></a></td>
      <td>OpenAI&#8209;compatible HTTP server. Drop&#8209;in for OpenAI clients.</td>
    </tr>
    <tr>
      <td><a href="https://github.com/oobabooga/text-generation-webui"><b>text&#8209;generation&#8209;webui</b></a></td>
      <td>Local chat UI. Pick the <i>ExLlamaV3</i> loader from the model dropdown.</td>
    </tr>
    <tr>
      <td><a href="https://github.com/turboderp-org/exllamav3"><b>ExLlamaV3</b></a></td>
      <td>Direct Python API for embedding the model in your own code or pipeline.</td>
    </tr>
  </tbody>
</table>

> [!TIP]
> **VRAM at 4.5&nbsp;bpw:** weights on disk + ~2&nbsp;GB context overhead. Comfortable on a single 24&nbsp;GB card with room for ~16k tokens of context; fits a 16&nbsp;GB card with a reduced context window.

## Download

```bash
pip install -U huggingface_hub

hf download \
  blockblockblock/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-exl3-4.5bpw \
  --local-dir ./Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-exl3-4.5bpw
```

<details>
<summary><b>Quantization recipe</b> &nbsp;<sub>(advanced &mdash; embedded in <code>quantization_config.json</code>)</sub></summary>

<br/>

| Setting | Value |
| :--- | :--- |
| Format | `EXL3` |
| Bits per weight | `4.5` |
| Head bits | `8` |
| Calibration rows | `250` |
| Codebook | `MCG` |
| Out&#8209;scales | `always` |
| Parallel mode | `enabled` (MoE expert batching) |

Loaded automatically by every ExLlamaV3 loader; reproduced here for searchability.

</details>

## License &amp; use

> [!IMPORTANT]
> Use and license **follow the [base model](https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled)**.
> Quantization adds no additional restrictions. Refer to the upstream repository for terms, citation, and safety documentation.

---

<div align="center">
  <sub><i>Quantized with <a href="https://github.com/Honkware/blockquant"><b>BlockQuant</b></a> &nbsp;·&nbsp; convention&nbsp;<code>{org}/{model}-exl3-{bpw}bpw</code></i></sub>
</div>