AtomicChat/Qwen3.8-Flash-Next-GGUF

How to Run Qwen3.8-Flash-Next Locally

Built from Qwen's original weights with our own importance matrix. The calibration corpora behind our builds are public.

• Qwen3.8-Flash-Next is the first open-weight release of the architecture behind Qwen4.
• These GGUFs are self-quantized from Qwen's original weights with our own importance matrix, published alongside the quants.
• The quants are still uploading and need a llama.cpp build with Qwen3.8-Flash-Next support; Atomic Chat runs it as support ships.

---

Qwen3.8-Flash-Next architecture (Qwen).

Highlights

• 125B total with 6B active sparse MoE (512 experts, 10 routed + 1 shared), plus a 51B n-gram embedding and a 4B MTP layer. An experimental preview of the architecture behind Qwen4.
• Hybrid attention with QSA: Gated DeltaNet paired with Qwen Sparse Attention, which operates at the micro-block level rather than per token to cut long-context latency for agentic workloads.
• Gated Residual: a data-dependent read gate plus a per-branch scalar write gate over widened residual streams, for finer expressiveness at low inference overhead.
• N-gram Embedding: 20M bigram/trigram embeddings indexed at layer 2, a compute-light axis for parameter scaling that offloads well on memory-constrained accelerators.
• 262,144-token context, extensible up to 1,000,000 tokens with RoPE scaling.
• Natively multimodal (causal language model with a vision encoder, image-text-to-text). These GGUF quants cover the text path.
• Frontier coding and agentic scores (Qwen-reported): LiveCodeBench v6 91.9, GPQA Diamond 91.7, SWE-bench Multilingual 81.0, CoWorkBench 73.9.
• Full imatrix quantization with our public calibration corpora.

These GGUFs are self-quantized from the original weights, not a repack. The importance matrix keeps low-bit quants closer to the full-precision model.

Always pass --jinja so the Qwen3.8-Flash-Next chat template is applied. Without it the model can emit malformed turns.

Model Overview

Property	Value
Base model	Qwen/Qwen3.8-Flash-Next
Type	Causal language model with a vision encoder (image-text-to-text)
Total / active parameters	125B total / 6B active, plus 51B n-gram embedding and a 4B MTP layer
Layers	48. Hidden layout: 12 x (3 x (Gated DeltaNet then MoE) then 1 x (Qwen Sparse Attention then MoE))
Experts	512 experts, 10 routed + 1 shared activated
Attention	Hybrid: Gated DeltaNet (linear) and Qwen Sparse Attention (micro-block sparse); Gated Residual over widened residual streams
Context length	262,144 native, extensible up to 1,000,000
This repo	GGUF quants (imatrix), text path. The importance matrix we built is published here too.

Scores are Qwen's published results for the base Qwen/Qwen3.8-Flash-Next. Quantization preserves the large majority of this; Q4_K_M and up sit within a point or two of full precision.

Choosing a quant

Quant	Size	Notes
IQ2_M	—	Smallest usable. Aggressive low-bit for memory-constrained boxes.
IQ3_M	—	Beats Q3 at similar size thanks to imatrix. Best low-RAM pick.
Q4_K_M	—	Recommended default. Best balance of size, speed and quality.
UD-Q4_K_XL	—	Dynamic. Embeddings and output kept at Q8_0 for higher quality at a Q4 footprint.
Q6_K	—	Near lossless.
Q8_0	—	Effectively lossless, reference quality.

Sizes fill in once the quants finish uploading. Pick the largest file that fits your (V)RAM with room for context.

Get started

Qwen3.8-Flash-Next is a brand-new Qwen4-preview architecture (Gated DeltaNet, Qwen Sparse Attention, n-gram embedding). The quants in this repo are still uploading, and running them needs a llama.cpp build that has landed Qwen3.8-Flash-Next support. Until then, Atomic Chat is the easiest way to run it as support ships.

Run Qwen3.8-Flash-Next locally with:

• Atomic Chat: the easiest path. Open the app, search AtomicChat/Qwen3.8-Flash-Next-GGUF, pick a quant, hit Use this model.
• llama.cpp: llama-server -hf AtomicChat/Qwen3.8-Flash-Next-GGUF:Q4_K_M --jinja -c 8192
• Ollama: ollama run hf.co/AtomicChat/Qwen3.8-Flash-Next-GGUF:Q4_K_M
• LM Studio / Jan: search the repo id, download any quant.

Best practices

Parameter	Value
temperature	1.0
top_p	0.95
top_k	20
min_p	0.0

Qwen's recommended thinking-mode settings. For non-thinking (instruct) use temperature=0.7, top_p=0.80, top_k=20, presence_penalty=1.5. Allocate generous output length for agentic tasks.

Run in llama.cpp

git clone https://github.com/ggerganov/llama.cpp
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --target llama-cli llama-server

./llama.cpp/build/bin/llama-server \
    -hf AtomicChat/Qwen3.8-Flash-Next-GGUF:UD-Q4_K_XL \
    --jinja -ngl 99 -c 8192 -fa on

How these were made

1. Download Qwen/Qwen3.8-Flash-Next (original weights).
2. Convert to GGUF with a llama.cpp build that supports the Qwen3.8-Flash-Next architecture (Gated DeltaNet, Qwen Sparse Attention, n-gram embedding).
3. Build an importance matrix over our public calibration corpora.
4. Quantize the ladder with --imatrix; UD-Q4_K_XL additionally pins the token-embedding and output tensors to Q8_0.

License

Released by Qwen under the Qwen Community License 1.0. Quantized by Atomic Chat.