File size: 3,558 Bytes
930f428 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 | ---
library_name: mlx
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/main/LICENSE
pipeline_tag: image-text-to-text
base_model: Qwen/Qwen3.6-35B-A3B
tags:
- mlx
- qwen3.6
- moe
- q4
- apple-silicon
---
# mavis-ai/Qwen3.6-35B-MoE-Q4
This repository contains an MLX-ready Q4 quantized build of Qwen3.6-35B-A3B, prepared for local inference on Apple Silicon Macs.
This is not a newly fine-tuned model. It is a local MLX redistribution converted from the official Qwen3.6-35B-A3B checkpoint. No additional training or architecture-level modification has been applied.
Source revision: `995ad96eacd98c81ed38be0c5b274b04031597b0`.
## Important Notice
This package uses a controlled, Gemma4-like mixed quantization policy. It is not a plain "quantize every linear layer to Q4" build.
For the original model card, architecture details, intended usage, limitations, and evaluation information, refer to the official upstream model:
- Base model: <https://huggingface.co/Qwen/Qwen3.6-35B-A3B>
## Quantization
This package uses MLX affine quantization with a mixed layout intended to preserve quality-sensitive tensor groups:
- Source checkpoint: `Qwen/Qwen3.6-35B-A3B`
- Default language tensor quantization: MLX affine Q4
- Token embeddings: MLX affine Q6
- LM head: MLX affine Q6
- MoE routing gates and shared expert gates: BF16
- Vision / multimodal components: BF16
- Group size: `64`
- Mode: `affine`
The goal is to keep the model closer to the higher-precision routing and multimodal behavior expected from the original checkpoint while still reducing local storage and memory requirements. For this Q4 build, vocabulary-facing tensors are kept at MLX affine Q6.
## Model Notes
Qwen3.6-35B-A3B is a multimodal Mixture-of-Experts model with 35B total parameters and approximately 3B activated parameters. The local source checkpoint reports:
- Hidden size: `2048`
- Layers: `40`
- Experts: `256`
- Activated experts: `8` routed experts plus shared expert
- Context length: `262144`
## Usage
Install or update the MLX runtime used for Qwen3.6 / multimodal models:
```bash
pip install -U mlx mlx-lm mlx-vlm huggingface_hub hf_xet
```
Download the model:
```bash
hf download mavis-ai/Qwen3.6-35B-MoE-Q4 \
--local-dir ~/Models/mlx/Qwen3.6-35B-MoE-Q4
```
Run a quick text generation test:
```bash
python -m mlx_vlm.generate \
--model ~/Models/mlx/Qwen3.6-35B-MoE-Q4 \
--max-tokens 128 \
--temperature 0.0 \
--prompt "Answer briefly: what can you do?"
```
## Files
Recommended repository files:
```text
.gitattributes
LICENSE
README.md
chat_template.jinja
config.json
configuration.json
generation_config.json
merges.txt
model-00001-of-00004.safetensors
model-00002-of-00004.safetensors
model-00003-of-00004.safetensors
model-00004-of-00004.safetensors
model.safetensors.index.json
preprocessor_config.json
processor_config.json
tokenizer.json
tokenizer_config.json
video_preprocessor_config.json
vocab.json
```
## License
This repository redistributes a quantized derivative of Qwen3.6-35B-A3B, which is distributed under the Apache License 2.0. A copy of the license is included in the `LICENSE` file in this repository.
## Modification Notice
Compared with the official Qwen source checkpoint, this repository applies the following packaging modification:
```text
The source checkpoint was converted to MLX format and quantized with a controlled mixed Q4/Q6/BF16 policy for local MLX inference.
```
No fine-tuning, additional training, or architecture-level modification has been applied.
|