Add ROCmFP4 STRIX quant + model card

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Qwen3.6-27B-MTP-pi-reasoning-ROCmFP4-STRIX.gguf filter=lfs diff=lfs merge=lfs -text

Qwen3.6-27B-MTP-pi-reasoning-ROCmFP4-STRIX.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4ebb0989fa672c33fc79ce4d5d90e1de6cf265535d5fcf83a5cd2252037b239
+size 14986109152

README.md

@@ -0,0 +1,133 @@
+---
+license: apache-2.0
+base_model: bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF
+base_model_relation: quantized
+pipeline_tag: text-generation
+library_name: gguf
+tags:
+  - gguf
+  - rocmfp4
+  - qwen3.6
+  - mtp
+  - speculative-decoding
+  - strix-halo
+  - amd
+  - rocm
+  - quantized
+---
+
+# Qwen3.6-27B-MTP-pi-reasoning · ROCmFP4 (STRIX)
+
+A **ROCmFP4** 4-bit quant of [`bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF`](https://huggingface.co/bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF),
+built for AMD **Strix Halo** (Ryzen AI MAX+, `gfx1151`) with multi-token-prediction (MTP) self-speculative decoding.
+Quantized from the BF16 source with the `Q4_0_ROCMFP4_STRIX` quality preset.
+
+> ⚠️ **This is not a stock GGUF.** The `q4_0_rocmfp4` / `q4_0_rocmfp4_fast` tensor types only load in the
+> [charlie12345/rocmfp4-llama](https://github.com/charlie12345/rocmfp4-llama) fork of llama.cpp (branch `mtp-rocmfp4-strix`).
+> It will **not** load in upstream llama.cpp, LM Studio, Ollama, Jan, or koboldcpp.
+> Hugging Face's auto-parser may mislabel the file as "F16"; it is really a ~4.4 bpw 4-bit model.
+
+## What is ROCmFP4?
+
+ROCmFP4 is an AMD-focused 4-bit GGUF weight format from the fork above. It pairs a Codebook10 4-bit value table
+with finite unsigned E4M3 half-scales, in two layouts:
+
+- **`q4_0_rocmfp4`**: dual-scale (~4.5 bpw), used on precision-sensitive tensors (attention projections here).
+- **`q4_0_rocmfp4_fast`**: single-scale (~4.25 bpw), used on the bulk of the network for speed.
+
+The `_STRIX` preset is a tensor-aware mix. It protects what matters for coherence while keeping the body small and fast
+on the AMD ROCm/HIP and Vulkan paths.
+
+## This build
+
+| | |
+|---|---|
+| Format | ROCmFP4 4-bit (`Q4_0_ROCMFP4_STRIX`) |
+| Effective precision | **4.38 bpw** |
+| File size | ~14.0 GiB (14,986,109,152 bytes) |
+| Architecture | `qwen35` hybrid attention + SSM, 65 blocks (64 plus 1 MTP) |
+| Context | up to 262,144 tokens |
+| MTP | `nextn` draft head carried through (self-speculative) |
+| imatrix | none (plain preset quant) |
+| Vision | not included, language weights only (see note below) |
+
+**Tensor recipe (as quantized):**
+
+| Tensor group | Type |
+|---|---|
+| `token_embd` | `q6_K` |
+| attention K/V projections | `q4_0_rocmfp4` (dual-scale) |
+| FFN, `output` (lm-head), MTP `eh_proj`, rest | `q4_0_rocmfp4_fast` (single-scale) |
+| norms, SSM params (`ssm_*`), MTP norms | `f32` |
+
+## Quick start (llama.cpp fork)
+
+Build the fork first (see its README; Strix uses `scripts/build-strix-rocmfp4-mtp.sh`), then:
+
+```bash
+HSA_OVERRIDE_GFX_VERSION=11.5.1 \
+GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
+./build-strix-rocmfp4/bin/llama-server \
+  -m Qwen3.6-27B-MTP-pi-reasoning-ROCmFP4-STRIX.gguf \
+  -dev ROCm0 -ngl 999 \
+  -c 262144 -b 512 -ub 512 -fa on \
+  -ctk q4_0 -ctv q4_0 \
+  --spec-type draft-mtp \
+  --spec-draft-type-k q4_0 --spec-draft-type-v q4_0 \
+  --spec-draft-n-max 4 --spec-draft-n-min 0 \
+  --spec-draft-p-min 0.0 --spec-draft-p-split 0.10 \
+  --reasoning on --jinja
+```
+
+Drop the `--spec-*` flags to run without MTP, or the `--reasoning`/`--jinja` flags for plain completion.
+
+## Local performance
+
+Measured on a Framework Desktop / Ryzen AI MAX+ 395, Radeon 8060S (`gfx1151`), 128 GB unified memory, ROCm backend,
+fork build `4795079b0`. Numbers are local and depend on driver, context, and prompt.
+
+Raw throughput, `llama-bench` (`-ngl 999 -fa 1 -r 3`), no speculative decoding:
+
+| test | t/s |
+|---|---|
+| prefill (pp512) | **389.3 ± 2.4** |
+| decode (tg128) | **13.8 ± 0.03** |
+
+`llama-bench` cannot exercise MTP, so the decode figure above is raw single-token decode.
+MTP self-speculative decoding (the intended way to run this model) roughly doubles it, with the gain depending heavily on
+how predictable the output is. Measured on `llama-server` over 256-token generations:
+
+| Workload | Decode tok/s | Draft acceptance |
+|---|---|---|
+| Code generation | ~35.5 | ~74% (190/257) |
+| Short reasoning | ~34.9 | ~72% (88/123) |
+| Free-form reasoning | ~27.1 | ~51% (170/335) |
+
+So expect roughly **27 to 36 tok/s** with MTP (about 2x to 2.6x over raw decode), trending higher on code and structured output.
+
+## Reproduce
+
+```bash
+# source: the BF16 GGUF from bytkim
+./build-strix-rocmfp4/bin/llama-quantize \
+  Qwen3.6-27B-MTP-pi-reasoning-bf16.gguf \
+  Qwen3.6-27B-MTP-pi-reasoning-ROCmFP4-STRIX.gguf \
+  Q4_0_ROCMFP4_STRIX
+```
+
+## Vision
+
+The source model is vision-capable via the Qwen3.6 `mmproj-F16.gguf` sidecar, but **this repo ships language weights only**.
+To enable images, pair this GGUF with the matching `mmproj` from the
+[source repo](https://huggingface.co/bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF) and pass `--mmproj` to the fork.
+
+## Lineage & credits
+
+- **Base model:** [`Qwen/Qwen3.6-27B`](https://huggingface.co/Qwen/Qwen3.6-27B) (Qwen team)
+- **Fine-tune / BF16 source:** [`bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF`](https://huggingface.co/bytkim/Qwen3.6-27B-MTP-pi-reasoning-GGUF), a QLoRA SFT on agent trajectories with MTP and "pi-reasoning"
+- **Format & runtime:** [`charlie12345/rocmfp4-llama`](https://github.com/charlie12345/rocmfp4-llama) (llama.cpp fork, MIT)
+
+## License
+
+Apache 2.0, inherited from the upstream `Qwen3.6-27B` base model. You may use, modify, and redistribute this quant
+and its derivatives subject to that license.