--- license: apache-2.0 base_model: badtheorylabs/BTL-4 base_model_relation: quantized pipeline_tag: image-text-to-text language: - en - zh library_name: gguf tags: - gguf - rocmfp4 - strix-halo - gfx1151 - ryzen-ai-max - moe - qwen3.5 - vision - llama-cpp - rocm - amd - llama.cpp --- # BTL-4 — ROCmFP4 STRIX GGUF — AMD Ryzen AI Max+ 395 / Strix Halo / gfx1151 **First ROCmFP4 quantization of [`badtheorylabs/BTL-4`](https://huggingface.co/badtheorylabs/BTL-4) that exists anywhere** (verified against the Hub before publish). Built for AMD Ryzen AI Max+ / Strix Halo (**gfx1151**). BTL-4 is a **Qwen3.5 MoE vision** model: architecture `Qwen3_5MoeForConditionalGeneration` / `model_type: qwen3_5_moe`, **40 layers**, **256 experts / 8 active**, hidden 2048, shared-expert 512, vocab 248320. Upstream `text_config.mtp_num_hidden_layers: 0` — **there is no MTP head**. Do **not** enable speculative / MTP drafting against this file; a spec flag with no tensors is a silent garbage drafter. Same architecture family as KAT-Coder-V2.5-Dev (8-of-256 active-param shape), which is where ROCmFP4 STRIX already beat Q4_K_M on Strix Halo. This build reproduces that pattern on BTL-4. ## Files | File | Notes | |---|---| | `BTL-4-Q4_0_ROCMFP4_STRIX.gguf` | Text MoE trunk, recipe **105** (`Q4_0_ROCMFP4_STRIX`) | | `mmproj-BTL-4-F16.gguf` | Vision projector (F16), load with `-mm` / `--mmproj` | Single-shard: **17.39 GiB** text + **0.84 GiB** mmproj. Under the HF 50 GB file cap — no split. ## Measured A/B (gfx1151, 128 GB unified, ROCm) Equal conditions for both quants: - Binary: `charlie12345/ROCmFPX` Laguna Strix export **`6255cc8`** (`export: Laguna Strix ROCmFP4 recipe on top of charlie12345/ROCmFPX@3edc3d3`) - Runtime: `-dio`, `HSA_OVERRIDE_GFX_VERSION=11.5.1`, `GGML_HIP_ENABLE_UNIFIED_MEMORY=1`, `-ngl 999`, `--no-warmup`, `--ignore-eos` - **256-token** generations, nonce-prefixed prompts (prefix cache defeated; **`cache_n == 0`** asserted every run) - **3-run medians**; **Q4_K_M baseline run twice** (noise control) - Quality: greedy (`temp 0`, `top_k 1`), thinking disabled via chat template kwargs, 10 prompts ### Size | Artifact | Bytes | BPW (real) | |---|---:|---:| | Upstream BF16 (HF) | 70,242,700,904 | bf16 | | F16 GGUF intermediate | 69,376,637,024 | 16.01 | | **This ROCmFP4 STRIX** | **18,664,879,904** | **4.31** (dry-run + build; advertised ~4.49) | | Same-model Q4_K_M control | 21,166,757,664 | 4.88 | STRIX is **−11.8%** smaller than the Q4_K_M control. ### Decode throughput (tok/s, median of 3) | Context | Q4_K_M A | Q4_K_M B | **ROCmFP4 STRIX** | vs doubled baseline | |---|---:|---:|---:|---:| | ~8K prompt | 54.28 | 54.02 | **61.06** | **+12.8%** | | ~32K prompt | 46.59 | 46.72 | **51.83** | **+11.1%** | Prompt-eval medians (tok/s): STRIX 1146.8 @8K / 857.4 @32K; Q4_K_M ~1105 / ~835. ### Quality (10-prompt greedy battery) | | Score | |---|---:| | **ROCmFP4 STRIX** | **10 / 10** | | Q4_K_M | **10 / 10** | Equal quality, clear speed win, smaller file → **ship**. ## Recipe notes - Prefer **`Q4_0_ROCMFP4_STRIX` (105)** over `_STRIX_LEAN` (106): same speed class, better quality headroom on this fork’s prior Strix A/Bs. - Real dry-run BPW was **4.31**, not the type’s advertised ~4.49. Always read dry-run. - Converted from the official BF16 with the fork’s `convert_hf_to_gguf.py` (`Qwen3_5MoeForConditionalGeneration` + `--mmproj`). **No `--mtp`.** ## Launch (Strix Halo / gfx1151) ```bash env HSA_OVERRIDE_GFX_VERSION=11.5.1 \ GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \ llama-server \ -m BTL-4-Q4_0_ROCMFP4_STRIX.gguf \ --mmproj mmproj-BTL-4-F16.gguf \ -ngl 999 -dio --no-warmup --jinja \ -c 32768 --parallel 1 \ --temp 0.0 --top-k 1 ``` **Do not pass MTP / speculative draft flags.** Upstream has zero MTP layers. Requires a ROCmFP4-capable llama.cpp build (ROCmFPX / Laguna Strix recipe), not stock llama.cpp alone, for the ROCmFP4 tensor types. ## License Inherited from [`badtheorylabs/BTL-4`](https://huggingface.co/badtheorylabs/BTL-4) (Apache-2.0 on the base card at publish time). All credit to the base authors; this repo is a quantization only (`base_model_relation: quantized`). ## Other public builds of this model Compiled from Hugging Face repository metadata — file sizes, shipped files, quant variant as named by each repo. **No third-party build was run or benchmarked here, so this table makes no speed or quality claim about any of them.** It is here so you can see the size and format options at a glance and pick what fits your hardware. | Repository | Largest model file | Variant | Ships | Downloads | Likes | | --- | ---: | --- | --- | ---: | ---: | | [`kingjones777/BTL-4-ROCmFP4-STRIX_LEAN-GGUF`](https://huggingface.co/kingjones777/BTL-4-ROCmFP4-STRIX_LEAN-GGUF) | 17.32 GiB | STRIX_LEAN | vision | 102 | 0 | | [`kingjones777/BTL-4-ROCmFP4-STRIX-GGUF`](https://huggingface.co/kingjones777/BTL-4-ROCmFP4-STRIX-GGUF) **(this repo)** | 17.38 GiB | STRIX | vision | 77 | 0 | *Base model: [`badtheorylabs/BTL-4`](https://huggingface.co/badtheorylabs/BTL-4). Generated from Hub metadata; download counts move over time.* ## Acknowledgements This build would not exist without the work below. Please star and follow these projects — the quantisation format used here is their engineering, not mine. **[ROCmFPX](https://github.com/charlie12345/ROCmFPX) — maintained by [`charlie12345`](https://github.com/charlie12345) / `caf`** The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every ROCmFP4 file in this repository was produced with its `llama-quantize`, and runs on its runtime. The fork also credits collaborators **ciru-ai**, **Tom Turney**, **PlunderStruck** and **Aydan S.**, and acknowledges AMD for hardware support. Licensed MIT, based on upstream llama.cpp. **[llama.cpp](https://github.com/ggml-org/llama.cpp) — ggml-org and contributors** The inference engine, GGUF format and conversion tooling everything here is built on. **[AMD ROCm](https://github.com/ROCm/ROCm)** The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S. **Base model authors** — see `base_model` in the metadata above; all model weights, licences and capabilities are theirs. This repository contributes quantisation and measurement only. If you use these files, please credit ROCmFPX alongside this repository.