--- license: apache-2.0 base_model: ibm-granite/granite-4.2-3b base_model_relation: quantized pipeline_tag: text-generation library_name: gguf tags: [gguf, llama.cpp, rocm, amd, strix-halo, gfx1151, ryzen-ai-max-395, rocmfp4, rocmfpx, strix-lean, granite, quantized] --- # Granite 4.2-3B (STRIX_LEAN) — ROCmFP4 for AMD Strix Halo (gfx1151) I built this STRIX_LEAN quantization of **ibm-granite/granite-4.2-3b** on my Strix Halo box for the ROCmFPX runtime. This is the 4th tier of my Granite 4.2 set — the lean 4-bit one people normally want. ## The file | | | |---|---| | ftype | `106` — `Q4_0_ROCMFP4_STRIX_LEAN` | | size | **2,066,204,736 bytes** (1.92 GiB) | | bpw | 4.51 | | architecture | `granite` | | tensors | 363 | | context | 131,072 | | token embedding | Q5_K (the LEAN part) | | `output.weight` | **Q6_K** (protected) | | sha256 | `72c0e6361a3c71c0d917d04b6cd576cebedabbe3f1799a3542fdcde6a3987c99` | Type histogram, read from the finished file: ``` Q4_0_ROCMFP4_FAST x200, F32 x81, Q4_0_ROCMFP4 x80, Q6_K x1, Q5_K x1 ``` ## What STRIX_LEAN is — and what it protects STRIX_LEAN is my lean 4-bit tier. The body is ROCmFP4 with the Strix Halo attention K/V quality recipe (that is what the STRIX part buys you), and the token embedding table is trimmed to **Q5_K** — that is the LEAN part, the size saving versus my COHERENT tier, which keeps the embeddings at Q6_K. What never gets trimmed is the head. Every STRIX_LEAN I publish carries the protected Q6_K LM head. This model has `tie_word_embeddings: false`, so `output.weight` is a real standalone tensor, and a 4-bit head would degrade the logits of every single token. I quantized with `--output-tensor-type q6_K` and confirmed the head landed at Q6_K by **exact**-name read-back on the finished file (`output.weight` — exact match, not substring). ## How I built it 1. Manifest gate: pulled `ibm-granite/granite-4.2-3b` file list from the HF API with `?blobs=true` and recorded the real shard bytes (2 safetensors shards, 7,319,517,120 bytes total — never the index `total_size`). 2. Downloaded and byte-verified **all 15 files** against that manifest (sizes + LFS sha256). 3. Converted with `convert_hf_to_gguf.py` from my `rocmfpx-dspark-halo` tree (4eca07e), `--outtype bf16` → 363 tensors, 7,323,461,696 bytes. 4. Quantized with the same tree's `llama-quantize` at 16 threads with `--output-tensor-type q6_K`. Dry-run estimate 1,967.08 MiB (4.51 bpw); the real file landed within ~3.5 MiB of it. ## Measured on my box — full GPU offload amd-halo: AMD Ryzen AI Max+ 395 (Strix Halo, gfx1151), ROCm 7.13.0, 128 GiB unified memory. Functional check at **full offload** — server flags `-dev ROCm0 -fa on -ngl 999 --no-mmap -fit off -np 1 -b 2048 -c 8192 -t 16 --jinja`, port 8497, greedy. 8 other llama-server seats were live on this machine while I tested (MemAvailable 16.3 GiB before load → 13.2 GiB after), so this is a functional check, not an idle-box benchmark. | | | |---|---:| | offload | **FULL — server log: `offloaded 41/41 layers to GPU`**, GTT usage +2.94 GB on load | | generation (server-reported) | **60.69 t/s** over 128 tokens | | prompt processing | 19 tokens in 69.2 ms | Sample output (greedy, prompt *"Explain in one clear sentence what granite rock is primarily made of."*): > Answer: Granite rock is primarily made of quartz. … (continued in the model's native self-check scaffold — real, structured generation) ## ⚠️ Stock llama.cpp will not load this file `Q4_0_ROCMFP4_STRIX_LEAN` is a custom tensor format that exists only in the [ROCmFPX](https://github.com/charlie12345/ROCmFPX) fork of llama.cpp. ```bash llama-server -m granite-4.2-3b-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev ROCm0 -fa on -ngl 999 -c 8192 ``` ## Not measured No benchmark sweeps, no context sweeps, no perplexity — one full-offload functional check, per my build discipline. ## Provenance & license Converted and quantized from `ibm-granite/granite-4.2-3b` (Apache 2.0). This quantized build is released under the same Apache 2.0 license. The ROCmFPX runtime is a third-party fork; its own terms apply to the runtime, not to these weights. ## All my quants of Granite-4.2-3B | build | what it is | size | tok/s (full GPU offload) | |---|---|---:|---:| | [`STRIX_LEAN`](https://huggingface.co/kingjones777/Granite-4.2-3B-ROCmFP4-STRIX_LEAN-GGUF) | my leaner 4-bit tier, Q6_K head — smallest of my 4-bit builds, the one most people want | 1.92 GiB | 60.69 | | [`COHERENT`](https://huggingface.co/kingjones777/Granite-4.2-3B-ROCmFP4-COHERENT-GGUF) | my 4-bit ROCmFP4 tier with the Q6_K-protected head — the balance I run day to day | 2.04 GiB | 71.55 | | [`Q8_0`](https://huggingface.co/kingjones777/Granite-4.2-3B-ROCmFPX-Q8_0-GGUF) | straight 8-bit ROCmFPX — highest fidelity I publish | 3.52 GiB | 46.36 | | [`Q8_0-AGENT`](https://huggingface.co/kingjones777/Granite-4.2-3B-ROCmFPX-Q8_0-AGENT-GGUF) | 8-bit ROCmFPX with the agent-tuned tensor set — for tool-calling work where precision matters | 3.59 GiB | 50.47 | All measured by me on a Ryzen AI MAX+ 395 (Strix Halo, gfx1151, ROCm 7.2.4) with the whole model on GPU (`-ngl 999`), 128-token greedy generation. A dash means I haven't measured that one yet — I won't put a number in a card I didn't measure. Base model: [ibm-granite/granite-4.2-3b](https://huggingface.co/ibm-granite/granite-4.2-3b)