Vontra/Qwen3.8-Flash-Next-MLX-oQ6

Qwen3.8 Flash Next — MLX oQ6

A mixed-precision MLX conversion of Qwen/Qwen3.8-Flash-Next, quantised directly from the official BF16 checkpoint.

Original model · Qwen overview · MLX-VLM · Qwen Community License 1.0

About this conversion

This repository contains an oQ6 mixed-precision MLX conversion produced directly from Qwen's BF16 weights. Layer sensitivity was measured with a validated 4-bit proxy; final tensor quantisation reread the official BF16 checkpoint. Group size 32 supports the model's 160-wide hashed n-gram embedding tables.

Item	Value
Base model	Qwen/Qwen3.8-Flash-Next
Format	MLX safetensors
Quantisation	oQ6 mixed precision; 6-bit affine base with protected modules at 6/8-bit
Base group size	32
Weight shards	31
Weight size	155.78 GB (145.08 GiB)
Configured context	262,144 tokens
Architecture	qwen4_exp vision-language sparse MoE

The upstream tokenizer, chat template, vision processor, and generation configuration are preserved. This base release does not include the optional native MTP head; an explicitly named MTP build requires matching qwen4_exp MTP runtime support and is being handled separately.

Qwen3.8 Flash Next uses the new qwen4_exp architecture. Use an oMLX or MLX-VLM build that explicitly lists qwen4_exp support. Older MLX-VLM releases cannot load this checkpoint.

Do not attach a Qwen3.8 27B MTP drafter to this model. The hidden sizes differ and the drafter is incompatible with Flash Next.

Quick start

hf download Vontra/Qwen3.8-Flash-Next-MLX-oQ6 \
  --local-dir Qwen3.8-Flash-Next-MLX-oQ6

With a compatible MLX-VLM runtime:

python -m mlx_vlm.generate \
  --model Qwen3.8-Flash-Next-MLX-oQ6 \
  --prompt "Explain sparse mixture-of-experts routing." \
  --max-tokens 512

Measured performance

Validated on an Apple M3 Studio with deterministic text-only generation after model load:

Test path	Result
Standalone MLX exact-copy smoke test	23.1 tokens/s
Standalone MLX, 142-token explanatory response	21.1 tokens/s
oMLX server, warmed 512-token response	19.5 tokens/s

The first oMLX request reported 22.77 seconds to load the model; that one-off load time is separate from generation speed. The 512-token server run is the most representative sustained result. Results vary with prompt length, cache state, sampling settings, runtime version, and memory pressure.

Architecture

Qwen3.8 Flash Next combines Gated DeltaNet, Qwen Sparse Attention, sparse mixture-of-experts layers, widened gated residual streams, and hashed bigram/trigram embeddings.

Architecture detail	Upstream value
Language-model parameters	125B total / 6B active
N-gram embedding	51B parameters
Layers	48
Routed / active experts	512 / 10, plus 1 shared
Attention heads / KV heads	24 / 2
Hidden size	2,560
Native configured context	262,144 tokens

For upstream evaluations, intended use, limitations, safety guidance, and the complete architecture discussion, see the original model card.

Conversion and validation

• Source: official BF16 checkpoint.
• Sensitivity-guided mixed-precision allocation: 6-bit base with 228 protected modules at 6/8-bit.
• All 3,671 converted tensors and 31 indexed shards were checked locally.
• Deterministic exact-copy, explanatory, and sustained 512-token generation tests passed on Apple silicon.
• The release payload was scanned for credentials, personal contact details, private paths, private network information, logs, caches, and private organisation data.

This is a community conversion, not an official Qwen release.

License and attribution

The upstream model is released under the Qwen Community License 1.0. The required licence text is included in this repository.

Model design, training, evaluations, and upstream documentation belong to Qwen and the original contributors. The MLX conversion, Apple-silicon validation, and packaging are provided by Vontra.