---
library_name: mlx
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/main/LICENSE
pipeline_tag: image-text-to-text
base_model: Qwen/Qwen3.6-35B-A3B
tags:
- mlx
- qwen3.6
- moe
- q4
- apple-silicon
---

# mavis-ai/Qwen3.6-35B-MoE-Q4

This repository contains an MLX-ready Q4 quantized build of Qwen3.6-35B-A3B, prepared for local inference on Apple Silicon Macs.

This is not a newly fine-tuned model. It is a local MLX redistribution converted from the official Qwen3.6-35B-A3B checkpoint. No additional training or architecture-level modification has been applied.

Source revision: `995ad96eacd98c81ed38be0c5b274b04031597b0`.

## Important Notice

This package uses a controlled, Gemma4-like mixed quantization policy. It is not a plain "quantize every linear layer to Q4" build.

For the original model card, architecture details, intended usage, limitations, and evaluation information, refer to the official upstream model:

- Base model: <https://huggingface.co/Qwen/Qwen3.6-35B-A3B>

## Quantization

This package uses MLX affine quantization with a mixed layout intended to preserve quality-sensitive tensor groups:

- Source checkpoint: `Qwen/Qwen3.6-35B-A3B`
- Default language tensor quantization: MLX affine Q4
- Token embeddings: MLX affine Q6
- LM head: MLX affine Q6
- MoE routing gates and shared expert gates: BF16
- Vision / multimodal components: BF16
- Group size: `64`
- Mode: `affine`

The goal is to keep the model closer to the higher-precision routing and multimodal behavior expected from the original checkpoint while still reducing local storage and memory requirements. For this Q4 build, vocabulary-facing tensors are kept at MLX affine Q6.

## Model Notes

Qwen3.6-35B-A3B is a multimodal Mixture-of-Experts model with 35B total parameters and approximately 3B activated parameters. The local source checkpoint reports:

- Hidden size: `2048`
- Layers: `40`
- Experts: `256`
- Activated experts: `8` routed experts plus shared expert
- Context length: `262144`

## Usage

Install or update the MLX runtime used for Qwen3.6 / multimodal models:

```bash
pip install -U mlx mlx-lm mlx-vlm huggingface_hub hf_xet
```

Download the model:

```bash
hf download mavis-ai/Qwen3.6-35B-MoE-Q4 \
  --local-dir ~/Models/mlx/Qwen3.6-35B-MoE-Q4
```

Run a quick text generation test:

```bash
python -m mlx_vlm.generate \
  --model ~/Models/mlx/Qwen3.6-35B-MoE-Q4 \
  --max-tokens 128 \
  --temperature 0.0 \
  --prompt "Answer briefly: what can you do?"
```

## Files

Recommended repository files:

```text
.gitattributes
LICENSE
README.md
chat_template.jinja
config.json
configuration.json
generation_config.json
merges.txt
model-00001-of-00004.safetensors
model-00002-of-00004.safetensors
model-00003-of-00004.safetensors
model-00004-of-00004.safetensors
model.safetensors.index.json
preprocessor_config.json
processor_config.json
tokenizer.json
tokenizer_config.json
video_preprocessor_config.json
vocab.json
```

## License

This repository redistributes a quantized derivative of Qwen3.6-35B-A3B, which is distributed under the Apache License 2.0. A copy of the license is included in the `LICENSE` file in this repository.

## Modification Notice

Compared with the official Qwen source checkpoint, this repository applies the following packaging modification:

```text
The source checkpoint was converted to MLX format and quantized with a controlled mixed Q4/Q6/BF16 policy for local MLX inference.
```

No fine-tuning, additional training, or architecture-level modification has been applied.