--- license: apache-2.0 license_link: https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/LICENSE thumbnail: https://huggingface.co/AtomicChat/Qwen3.6-27B-UDT-MTP-GGUF/resolve/main/hero.png base_model: - Qwen/Qwen3.6-27B base_model_relation: quantized quantized_by: AtomicChat pipeline_tag: image-text-to-text library_name: gguf tags: - atomic-chat - qwen3.6 - qwen - gguf - llama.cpp - quantized ---

Base model: Qwen/Qwen3.6-27B

**Qwen3.6 27B**, self-quantized to GGUF by [Atomic Chat](https://atomic.chat). Built straight from Qwen's original weights with a per-tensor importance matrix, so this is not a repack of somebody else's files. Runs fully offline. ## Highlights - **27.8B parameters**: the weights this repo quantizes. - **Context length**: 262,144 tokens (256K), as published by Qwen. - **64 layers**: Dense decoder. - **Modalities**: Text, Image. - **Full imatrix ladder**: every quant is calibrated with an importance matrix. - **Agentic Coding:**: the model now handles frontend workflows and repository-level reasoning with greater fluency and precision. - **Thinking Preservation:**: we've introduced a new option to retain reasoning context from historical messages, streamlining iterative development and reducing overhead. > [!NOTE] > These GGUFs are **self-quantized from the original weights**, not a repack. The importance matrix keeps low-bit quants closer to the full-precision model. > [!IMPORTANT] > Always pass `--jinja` so the **Qwen3.6 27B chat template** is applied. Without it the model can emit malformed turns. ## Model Overview | Property | Value | |---|---| | Base model | `Qwen/Qwen3.6-27B` | | Parameters | 27.8B | | Layers | 64 | | Context length | 262,144 tokens (256K) | | Vocabulary | 248,320 | | Modalities | Text, Image | | Architecture | Dense decoder, 24 attention heads over 4 KV heads, `Qwen3_5ForConditionalGeneration` | | This repo | GGUF quants (imatrix) and a vision mmproj | > [!NOTE] > Qwen3.6 27B is multimodal. This repo ships the **`mmproj-BF16.gguf`** vision projector. With `-hf` it is pulled automatically; otherwise pass `--mmproj`. Use `llama-mtmd-cli` or `llama-server` to feed images. ## Get started Run Qwen3.6 27B locally with: - **[Atomic Chat](https://atomic.chat):** the easiest path. Open the app, search `AtomicChat/Qwen3.6-27B-UDT-MTP-GGUF`, pick a quant, hit **Use this model**. - **llama.cpp:** `llama-server -hf AtomicChat/Qwen3.6-27B-UDT-MTP-GGUF:None --jinja -c 8192` - **Ollama:** `ollama run hf.co/AtomicChat/Qwen3.6-27B-UDT-MTP-GGUF:None` - **LM Studio / Jan:** search the repo id, download any quant. ## Best practices | Parameter | Value | |---|---| | temperature | 1.0 | | top_p | 0.95 | | top_k | 20 | | min_p | 0.0 | | repetition_penalty | 1.0 | Qwen's recommended sampling configuration for `Qwen/Qwen3.6-27B`. Pass images through `llama-mtmd-cli` or `llama-server` with the projector. ## Run in llama.cpp ```bash git clone https://github.com/ggml-org/llama.cpp cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build llama.cpp/build --config Release -j --target llama-cli llama-server ``` ```bash ./llama.cpp/build/bin/llama-server \ -hf AtomicChat/Qwen3.6-27B-UDT-MTP-GGUF:None \ --jinja -ngl 99 -c 8192 -fa on ``` ## How these were made 1. Download `Qwen/Qwen3.6-27B` (original weights). 2. Convert to f16 GGUF with [llama.cpp](https://github.com/ggml-org/llama.cpp). 3. Build an importance matrix over our calibration corpus. 4. Quantize the ladder with `--imatrix`. ## License Original model by Qwen, released under the Apache 2.0 license. Full terms: [Apache 2.0](https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/LICENSE). Quantized by Atomic Chat.