--- license: apache-2.0 base_model: lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled base_model_relation: quantized library_name: mlx pipeline_tag: image-text-to-text language: - en - zh tags: - mlx - mlx-vlm - apple-silicon - oq - oq8 - quantized - 8-bit - moe - mixture-of-experts - vision-language - multimodal - qwen - qwen3.6 - reasoning - chain-of-thought - distillation - claude --- # Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ8 Mixed-precision quantization for Apple Silicon, with **vision-language (VLM) capabilities preserved**. **Highest quality version** in the series — closest to BF16 baseline. Quantized from [`lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled`](https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled) using [oMLX](https://omlx.app)'s oQ8 algorithm (sensitivity-aware mixed-precision quantization). ## 📊 Specs | Field | Value | |---|---| | **Base model** | Qwen/Qwen3.6-35B-A3B (35B params, 128 experts MoE, A3B activation) | | **Fine-tune** | LoRA distilled from Claude 4.7 Opus reasoning outputs (lordx64 dataset) | | **Quantization** | oMLX oQ8 (mixed-precision, ~8.7 bpw average) | | **Modality** | Vision + Text (VLM) | | **Format** | MLX safetensors | | **Model size** | ~35 GB | | **Inference memory** | ~39 GB (incl. KV cache and runtime overhead) | | **Recommended hardware** | Apple Silicon M2 Ultra 64GB+ / M3 Max / M5 Max | ## 🚀 Quick Start ### Install ```bash pip install mlx-vlm # Or with uv: uv tool install mlx-vlm --with torch --with torchvision ``` ### Inference (image + text) ```bash mlx_vlm.generate \ --model wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ8 \ --image /path/to/image.jpg \ --prompt "Describe this image in detail." \ --max-tokens 256 ``` ### Python API ```python from mlx_vlm import load, generate model, processor = load( "wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ8" ) output = generate( model, processor, image="/path/to/image.jpg", prompt="What's in this image?", max_tokens=512, ) print(output) ``` ### OpenAI-compatible Server ```bash mlx_vlm.server \ --model wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ8 \ --port 8080 ``` Drop-in compatible with OpenAI clients including AstrBot, Open WebUI, LibreChat, and Continue.dev. ## 📈 Measured Performance Benchmarked on MacBook Pro M5 Max 128GB: | Metric | Value | |---|---| | Prompt processing | ~637 tokens/s | | Generation speed | ~96 tokens/s | | Peak memory | 38.7 GB | | Model load time | ~14 sec | ### When to Choose oQ8 over oQ6 oQ8 retains slightly more precision than oQ6, but the **observable quality difference is small** for most tasks. Use oQ8 when: - You need the absolute highest fidelity quantization - Running quality benchmarks against the BF16 reference - Memory budget is generous (39+ GB free) For most users, [oQ6](https://huggingface.co/wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ6) is the better choice — comparable quality at 8 GB less footprint. ## 🧠 Model Behavior Inherits the Claude reasoning distillation: the model uses `...` tags to structure its chain-of-thought before producing the final response. **Best for:** - Multimodal reasoning tasks (image analysis with complex thinking) - Agentic workflows / tool use - Quality-sensitive applications where size is not a constraint - Quantization quality reference / baseline comparisons ## 🔬 Quantization Details - **Source model**: `Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled` (BF16 MLX-converted) - **Sensitivity model**: 8-bit quantization of the same distilled model (self-referenced sens for tight distribution alignment) - **Non-quant weight dtype**: bfloat16 (M3+ optimal) - **Text-Only mode**: OFF (vision tower preserved) - **Quantizer**: oMLX with mlx-vlm conversion path The vision processor configurations (`preprocessor_config.json`, `video_preprocessor_config.json`, `processor_config.json`) were sourced from the official Qwen base model to ensure proper image input handling — these were missing from the upstream distilled checkpoint. ## 📦 Other Versions in This Series | Version | Size | Best for | |---|---|---| | [VLM-MLX-oQ4](https://huggingface.co/wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ4) | 19.6 GB | Memory-constrained inference | | [VLM-MLX-oQ6](https://huggingface.co/wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ6) | 27 GB | **Recommended: best quality/size ratio** | | [VLM-MLX-oQ8](https://huggingface.co/wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ8) | 35 GB | Quality reference baseline | | [Text-MLX-oQ4](https://huggingface.co/wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-Text-MLX-oQ4) | 19 GB | Text-only, fastest | | [Text-MLX-oQ6](https://huggingface.co/wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-Text-MLX-oQ6) | 27 GB | Text-only, balanced | | [Text-MLX-oQ8](https://huggingface.co/wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-Text-MLX-oQ8) | 34 GB | Text-only, max quality | **Choosing a version:** - Text-only workflows (coding, agents, dialogue) → `Text` variants are faster and lighter - Image input needed (OCR, visual analysis, screenshot understanding) → `VLM` variants - oQ6 is the sweet spot for most use cases. oQ8 yields diminishing returns relative to its size. ## ⚠️ Disclaimer This model derives from a chain of upstream work: 1. Base model [`Qwen/Qwen3.6-35B-A3B`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) by Alibaba's Qwen team (Apache-2.0) 2. Distilled variant [`lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled`](https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled) by lordx64 using Claude 4.7 Opus reasoning outputs (Apache-2.0) 3. This quantization by [@wangkezun](https://huggingface.co/wangkezun) using oMLX on Apple Silicon **This model is not affiliated with or endorsed by Anthropic, PBC.** "Claude" is a trademark of Anthropic, PBC. The use of "Claude" in this model name is purely descriptive (nominative fair use) to indicate the upstream training data lineage. By using this model, you agree to comply with: - The Apache-2.0 license inherited from the base model - Any applicable license terms of the upstream distillation dataset - Local laws and regulations governing AI model usage in your jurisdiction ## 🙏 Acknowledgments - **Alibaba Qwen Team** — for the Qwen3.6-35B-A3B base model - **lordx64** — for the reasoning-focused LoRA distillation - **Jundot (oMLX team)** — for the oQ mixed-precision quantization algorithm - **Apple MLX team** — for the MLX framework and tooling - **mlx-vlm contributors** — for the VLM conversion path ## 📜 License Apache-2.0 (inherited from base model). --- **Generated**: 2026-04-26 **Quantizer**: [@wangkezun](https://huggingface.co/wangkezun)