A mixed-precision MLX conversion of Qwen/Qwen3.8-Flash-Next, quantised directly from the official BF16 checkpoint.
Original model · Qwen overview · MLX-VLM · Qwen Community License 1.0
## About this conversion This repository contains an oQ6 mixed-precision MLX conversion produced directly from Qwen's BF16 weights. Layer sensitivity was measured with a validated 4-bit proxy; final tensor quantisation reread the official BF16 checkpoint. Group size 32 supports the model's 160-wide hashed n-gram embedding tables. | Item | Value | | --- | --- | | Base model | [`Qwen/Qwen3.8-Flash-Next`](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | | Format | MLX safetensors | | Quantisation | oQ6 mixed precision; 6-bit affine base with protected modules at 6/8-bit | | Base group size | 32 | | Weight shards | 31 | | Weight size | 155.78 GB (145.08 GiB) | | Configured context | 262,144 tokens | | Architecture | `qwen4_exp` vision-language sparse MoE | The upstream tokenizer, chat template, vision processor, and generation configuration are preserved. This base release does not include the optional native MTP head; an explicitly named MTP build requires matching `qwen4_exp` MTP runtime support and is being handled separately. > [!IMPORTANT] > Qwen3.8 Flash Next uses the new `qwen4_exp` architecture. Use an oMLX or MLX-VLM build that explicitly lists `qwen4_exp` support. Older MLX-VLM releases cannot load this checkpoint. > [!CAUTION] > Do not attach a Qwen3.8 27B MTP drafter to this model. The hidden sizes differ and the drafter is incompatible with Flash Next. ## Quick start ```bash hf download Vontra/Qwen3.8-Flash-Next-MLX-oQ6 \ --local-dir Qwen3.8-Flash-Next-MLX-oQ6 ``` With a compatible MLX-VLM runtime: ```bash python -m mlx_vlm.generate \ --model Qwen3.8-Flash-Next-MLX-oQ6 \ --prompt "Explain sparse mixture-of-experts routing." \ --max-tokens 512 ``` ## Measured performance Validated on an Apple M3 Studio with deterministic text-only generation after model load: | Test path | Result | | --- | ---: | | Standalone MLX exact-copy smoke test | 23.1 tokens/s | | Standalone MLX, 142-token explanatory response | 21.1 tokens/s | | oMLX server, warmed 512-token response | 19.5 tokens/s | The first oMLX request reported 22.77 seconds to load the model; that one-off load time is separate from generation speed. The 512-token server run is the most representative sustained result. Results vary with prompt length, cache state, sampling settings, runtime version, and memory pressure. ## Architecture Qwen3.8 Flash Next combines Gated DeltaNet, Qwen Sparse Attention, sparse mixture-of-experts layers, widened gated residual streams, and hashed bigram/trigram embeddings. | Architecture detail | Upstream value | | --- | ---: | | Language-model parameters | 125B total / 6B active | | N-gram embedding | 51B parameters | | Layers | 48 | | Routed / active experts | 512 / 10, plus 1 shared | | Attention heads / KV heads | 24 / 2 | | Hidden size | 2,560 | | Native configured context | 262,144 tokens | For upstream evaluations, intended use, limitations, safety guidance, and the complete architecture discussion, see the [original model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next). ## Conversion and validation - Source: official BF16 checkpoint. - Sensitivity-guided mixed-precision allocation: 6-bit base with 228 protected modules at 6/8-bit. - All 3,671 converted tensors and 31 indexed shards were checked locally. - Deterministic exact-copy, explanatory, and sustained 512-token generation tests passed on Apple silicon. - The release payload was scanned for credentials, personal contact details, private paths, private network information, logs, caches, and private organisation data. This is a community conversion, not an official Qwen release. ## License and attribution The upstream model is released under the **Qwen Community License 1.0**. The required licence text is included in this repository. Model design, training, evaluations, and upstream documentation belong to Qwen and the original contributors. The MLX conversion, Apple-silicon validation, and packaging are provided by [Vontra](https://huggingface.co/Vontra).