mavis-ai/Qwen3.6-35B-MoE-Q4

This repository contains an MLX-ready Q4 quantized build of Qwen3.6-35B-A3B, prepared for local inference on Apple Silicon Macs.

This is not a newly fine-tuned model. It is a local MLX redistribution converted from the official Qwen3.6-35B-A3B checkpoint. No additional training or architecture-level modification has been applied.

Source revision: 995ad96eacd98c81ed38be0c5b274b04031597b0.

Important Notice

This package uses a controlled, Gemma4-like mixed quantization policy. It is not a plain "quantize every linear layer to Q4" build.

For the original model card, architecture details, intended usage, limitations, and evaluation information, refer to the official upstream model:

• Base model: https://huggingface.co/Qwen/Qwen3.6-35B-A3B

Quantization

This package uses MLX affine quantization with a mixed layout intended to preserve quality-sensitive tensor groups:

• Source checkpoint: Qwen/Qwen3.6-35B-A3B
• Default language tensor quantization: MLX affine Q4
• Token embeddings: MLX affine Q6
• LM head: MLX affine Q6
• MoE routing gates and shared expert gates: BF16
• Vision / multimodal components: BF16
• Group size: 64
• Mode: affine

The goal is to keep the model closer to the higher-precision routing and multimodal behavior expected from the original checkpoint while still reducing local storage and memory requirements. For this Q4 build, vocabulary-facing tensors are kept at MLX affine Q6.

Model Notes

Qwen3.6-35B-A3B is a multimodal Mixture-of-Experts model with 35B total parameters and approximately 3B activated parameters. The local source checkpoint reports:

• Hidden size: 2048
• Layers: 40
• Experts: 256
• Activated experts: 8 routed experts plus shared expert
• Context length: 262144

Usage

Install or update the MLX runtime used for Qwen3.6 / multimodal models:

pip install -U mlx mlx-lm mlx-vlm huggingface_hub hf_xet

Download the model:

hf download mavis-ai/Qwen3.6-35B-MoE-Q4 \
  --local-dir ~/Models/mlx/Qwen3.6-35B-MoE-Q4

Run a quick text generation test:

python -m mlx_vlm.generate \
  --model ~/Models/mlx/Qwen3.6-35B-MoE-Q4 \
  --max-tokens 128 \
  --temperature 0.0 \
  --prompt "Answer briefly: what can you do?"

Files

Recommended repository files:

.gitattributes
LICENSE
README.md
chat_template.jinja
config.json
configuration.json
generation_config.json
merges.txt
model-00001-of-00004.safetensors
model-00002-of-00004.safetensors
model-00003-of-00004.safetensors
model-00004-of-00004.safetensors
model.safetensors.index.json
preprocessor_config.json
processor_config.json
tokenizer.json
tokenizer_config.json
video_preprocessor_config.json
vocab.json

License

This repository redistributes a quantized derivative of Qwen3.6-35B-A3B, which is distributed under the Apache License 2.0. A copy of the license is included in the LICENSE file in this repository.

Modification Notice

Compared with the official Qwen source checkpoint, this repository applies the following packaging modification:

The source checkpoint was converted to MLX format and quantized with a controlled mixed Q4/Q6/BF16 policy for local MLX inference.

No fine-tuning, additional training, or architecture-level modification has been applied.