---
license: other
base_model: meta-models/Muse-Glimmer-30B
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
tags:
- gguf
- llama.cpp
- rocm
- amd
- rocmfp4
- rocmfpx
- strix-halo
- amd-strix-halo
- gfx1151
- ryzen-ai-max
- ryzen-ai-max-395
- radeon-8060s
- dflash
- speculative-decoding
- tool-calling
- multimodal
- vision
- muse
- glimmer
- quantized
---

# Muse-Glimmer-30B — ROCmFPX 8-bit AGENT for AMD Strix Halo (gfx1151)

<!-- LEAD-CLAIM:START -->

> ✅ **the complete `muse-glimmer` port — text graph, vision projector and chat parser — ships as a single applyable patch in this repo**
>
> *`muse-glimmer` is not an upstream llama.cpp architecture. Running it end-to-end takes three independent pieces of work; all three are in `patches/muse-glimmer-complete.patch` (20 files, 81,968 bytes, `git apply --check` clean).*

<!-- LEAD-CLAIM:END -->

An 8-bit ROCmFPX quantisation of **Muse-Glimmer-30B** for AMD Ryzen AI Max+ 395 / Radeon 8060S / gfx1151, quantised from the BF16 GGUF — a lossless source, not a requantisation of a smaller file. ROCmFPX is a runtime tensor format that exists only in the [ROCmFPX](https://github.com/charlie12345/ROCmFPX) fork of llama.cpp.

| Metric | Result |
| --- | ---: |
| Quantization | `Q8_0_ROCMFPX_AGENT` (ggml ftype 115) |
| Model size | **27.23 GiB** (29,235,965,312 bytes) |
| Effective BPW | **8.39** (measured, not advertised) |
| Source | BF16 GGUF, 55,725,514,112 bytes |
| Tested hardware | AMD Ryzen AI Max+ 395 (Strix Halo), 128 GB unified |
| GPU | Radeon 8060S, gfx1151 |
| Decode, no draft head | **7.22 tok/s** (median of 3: 7.22, 7.21, 7.26) |
| Decode with DFlash, prose | 11.31 tok/s |
| Correctness | ✅ **3/3** — 391 / Tokyo / 366 |
| Vision (256×256 placement) | ✅ **pass**, requires `-fa off` |
| Chat parser | ✅ clean `content` — no control-token leak |
| Speculative head | **DFlash**, not MTP |

## Why this build?

- **Quantised from BF16**, source revision verified by fetch (`a4e59da52a7bc87ae7251dd5545c0dd437c44b68`) rather than assumed
- **Output head and token embeddings pinned to q8_0** explicitly, not left to inherit the block type
- **The full architecture port is in the repo** — you do not have to reconstruct it
- **Vision works** through the BF16 projector built by stage 2 of that patch
- Sizes are reported from `stat`, and checked against the `--dry-run` projection: the ~12.5 MiB delta is GGUF header, which is the signature of a complete file

## Which file should I use?

**If you want speed, take a 4-bit build** — on this model the 4-bit files are roughly **3.5× the code-transform throughput** of the 8-bits and less than half the size. The 8-bits are here for bit-count, not for speed.

| Build | ftype | Size | BPW | prose | code-transform | accept len (code) |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| `ROCmFP4-FAST` | 103 | 13.80 GiB | 4.25 | 15.07 | **39.35** | 7.12 |
| `ROCmFP4-STRIX` | 105 | 14.17 GiB | 4.39 | 14.96 | 37.55 | 6.80 |
| **`Q8_0_ROCMFPX`** | 111 | **26.85 GiB** | 8.28 | 11.31 | — | 2.65 |
| **`Q8_0_ROCMFPX_AGENT`** | 115 | **27.23 GiB** | 8.39 | 11.27 | — | 2.51 |

> ⚠️ **Decode on this model is workload-dominated, not variant-dominated.** DFlash proposes long runs on repetitive and code-like text and very little on freeform prose, so a single tok/s figure is misleading — mean accepted length moves **2.5 → 7.1** across the same binary and the same weights. Quote a range for this model, not a point.

`muse-glimmer-30B-Q8_0_ROCMFPX_AGENT.gguf` and [`muse-glimmer-30B-Q8_0_ROCMFPX.gguf`](https://huggingface.co/kingjones777/Muse-Glimmer-30B-ROCmFPX-Q8_0-GGUF) are within noise of each other (7.22 vs the other build's figure on the same harness). The `AGENT` routing lifts draft acceptance on **MTP** models; this model uses **DFlash**, so there is nothing for it to win here. Choose on size.

## Quick start

```bash
hf download kingjones777/Muse-Glimmer-30B-ROCmFPX-Q8_0-AGENT-GGUF --local-dir glimmer
```

The DFlash drafter is not duplicated here — pull it from the 4-bit repo, which carries every
drafter and projector variant:

```bash
hf download kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF \
  --include "dflash-ROCmFP4-STRIX.gguf" --local-dir glimmer
```

⚠️ `hf download` silently ignores `--include` when given more than one pattern — one call per file.

```bash
llama-server \
  -m glimmer/muse-glimmer-30B-Q8_0_ROCMFPX_AGENT.gguf \
  --mmproj glimmer/mmproj-muse-glimmer-30B-BF16.gguf \
  --spec-type draft-dflash --model-draft glimmer/dflash-ROCmFP4-STRIX.gguf \
  --spec-draft-ngl 99 \
  -ngl 999 -c 4096 -fa off -fit off --jinja \
  --host 127.0.0.1 --port 8080
```

### Three flags that matter more than which file you pick

| Flag | Why |
| --- | --- |
| `--model-draft` | **Serve it with the DFlash head.** Without one, the 8-bit build drops 11.62 → 7.65 tok/s (−34%). This single flag outweighs the quantisation choice. |
| `-fa off` | Required for the **vision** path on gfx1151. Text-only serving can use `-fa on`. |
| `-fit off` | llama.cpp's autofit reads `MemAvailable` on integrated GPUs, which is at its lowest right after a model unload — leaving it on can silently shrink context or push tensors to CPU. |

⚠️ This head is **DFlash, not MTP**. Read *mean accepted length*, and do not pass MTP flags to it.

## Vision

Verified on **spatial placement** rather than plausible-sounding output: a solid-colour image is scored on whether the model names the colour that is actually there.

| Input | Result |
| --- | ---: |
| 256×256 solid red | ✅ `red` |

Serve with **`-fa off`** and the BF16 projector from this repo (`mmproj-muse-glimmer-30B-BF16.gguf`, 3,849,174,048 bytes, 809 tensors).

⚠️ **Minimum useful image size is 28×28 px.** The preprocessor snaps to `patch 14 × merge 2`, so anything smaller collapses to a single merge token, carries no spatial signal, and the model reports the dominant colour of the padded canvas. Feed 256×256 or larger. `llama-mtmd-cli` behaves identically — this is the preprocessing geometry, not the projector.

## Files

| File | Size | Role |
| --- | ---: | --- |
| `muse-glimmer-30B-Q8_0_ROCMFPX_AGENT.gguf` | 27.23 GiB | model — this repo |
| `mmproj-muse-glimmer-30B-BF16.gguf` | 3.58 GiB | vision projector — **use this one** |
| `patches/muse-glimmer-complete.patch` | 80 KiB | 20-file architecture port |
| `dflash-ROCmFP4-STRIX.gguf` | 1.39 GiB | DFlash drafter — **lives in the [4-bit repo](https://huggingface.co/kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF)**, not here |

## 🩹 The muse-glimmer architecture port — complete, three stages

`patches/muse-glimmer-complete.patch` — **81,968 bytes, 20 files**, `git apply --check` clean
against [`charlie12345/ROCmFPX`](https://github.com/charlie12345/ROCmFPX).

```bash
git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
git apply --check ../patches/muse-glimmer-complete.patch && \
git apply           ../patches/muse-glimmer-complete.patch
cmake -S . -B build-muse -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 \
      -DLLAMA_BUILD_WEBUI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build-muse --target llama-server --target llama-quantize \
      --target llama-mtmd-cli -j 6
```

`muse-glimmer` is not an upstream architecture. Running it end-to-end takes three independent
pieces of work, and all three are in this patch.

### Stage 1 — text graph and conversion

| File | Role |
| --- | --- |
| `src/llama-arch.{h,cpp}` | `LLM_ARCH_MUSE_GLIMMER` + tensor-name table |
| `src/llama-model.cpp` | model factory case |
| `src/models/muse-glimmer.cpp` | the graph itself |
| `src/models/models.h` | declaration |
| `gguf-py/gguf/{constants,tensor_mapping}.py` | GGUF constants + tensor map |
| `conversion/{__init__,muse_glimmer}.py` | safetensors → GGUF converter |

⚠️ `convert_hf_to_gguf.py` does **not** know MuseGlimmer. The converter is
`conversion/muse_glimmer.py`, driven through `conversion.get_model_class`.

### Stage 2 — vision projector (clip / mtmd)

| File | Role |
| --- | --- |
| `tools/mtmd/models/muse-glimmer.cpp` | projector graph |
| `tools/mtmd/clip-impl.h` | `PROJECTOR_TYPE_MUSE_GLIMMER` |
| `tools/mtmd/clip-model.h` | hparams + LANCZOS enum |
| `tools/mtmd/clip-graph.h` | `build_vit_opts` + 7-arg overload |
| `tools/mtmd/clip.cpp`, `models/models.h`, `CMakeLists.txt` | wiring |
| `tools/mtmd/mtmd-image.{cpp,h}` | preprocessor; LANCZOS → bicubic_pillow fallback |
| `tools/mtmd/mtmd.cpp` | `<\|image_start\|>` / `<\|image_end\|>` markers |
| `gguf-py/gguf/tensor_mapping.py` | `model.vision_tower.layers.{bid}.attn.{q,k,v,proj}`, `norm1/2`, `mlp.fc1/2`, `ln_post` |
| `gguf-py/gguf/constants.py` | `VisionProjectorType.MUSE_GLIMMER` |

This stage is what makes `--mmproj` work. **Use the BF16 projector in this repo**
(`mmproj-muse-glimmer-30B-BF16.gguf`, **3,849,174,048 bytes**, 809 tensors,
`clip.projector_type = muse-glimmer`, merge 2, patch 14, image_size 896).

### Stage 3 — chat parser

`common/chat.cpp` — `common_chat_params_init_muse_glimmer`, PEG_NATIVE.

Without it the model's `to=self<|message|>` control sequence is emitted into `content`. With it,
`content` is clean: *"The capital of Japan is Tokyo."*

### Build note

`cmake`'s source GLOB is **configure-time**. After the patch adds `src/models/muse-glimmer.cpp` you
must re-run the `cmake -S . -B build-muse` configure step, not just `--build`.
`-DLLAMA_BUILD_WEBUI=OFF` avoids a node/npm requirement.

### Applying to a different base commit

The patch header names commit `3edc3d3`, and it applies cleanly to later revisions
(verified on `b41ce12`). On trees where `cohere2moe` and `bailing_hybrid` model sources are absent,
their factory cases in `llama-model.cpp` reference symbols that do not exist in that tree — build
those two out, or apply on `3edc3d3` where their `.cpp` files are present. The muse-glimmer factory
case and graph are independent of both.

## Quantization methodology

```bash
# 1. convert BF16 safetensors -> GGUF (the muse-glimmer converter is not in
#    convert_hf_to_gguf.py; it ships as conversion/muse_glimmer.py in the patch)
python -c "from conversion import get_model_class; ..."   # driver, see patch README

# 2. measure the real BPW before committing
llama-quantize --dry-run muse-glimmer-30B-BF16.gguf /tmp/x.gguf Q8_0_ROCMFPX 8

# 3. quantize with the patched ROCmFPX build (only it has ggml types 100-106)
llama-quantize --output-tensor-type q8_0 --token-embedding-type q8_0 \
  muse-glimmer-30B-BF16.gguf muse-glimmer-30B-Q8_0_ROCMFPX.gguf Q8_0_ROCMFPX 16
```

Source: `meta-models/Muse-Glimmer-30B`, revision
**`a4e59da52a7bc87ae7251dd5545c0dd437c44b68`** (fetched and verified, not assumed) →
`muse-glimmer-30B-BF16.gguf` **55,725,514,112 bytes**. Quantized from BF16 only — this is not a
requantization of a smaller file.

`--output-tensor-type` and `--token-embedding-type` are both set explicitly so the output head and
token embeddings land at q8_0 rather than inheriting the block type.

## Not yet measured

Listed so nobody mistakes absence for a pass:

| Test | Status |
| --- | --- |
| Perplexity / KL divergence vs BF16 | ❓ not measured |
| Long-context behaviour beyond 4096 | ❓ not measured |
| Vision beyond single-colour placement (OCR, charts, documents) | ❓ not measured |
| MMLU-Pro, GPQA, GSM8K, HumanEval+ | ❓ not run |
| Multi-step agentic loop | ❓ not run |
| Sustained multi-thousand-token generation | ❓ not measured |
| Independent reproduction | ❓ none yet |

## Known issues

1. **Stock llama.cpp cannot load these files.** `muse-glimmer` is not an upstream architecture and
   ROCmFP4/ROCmFPX are not upstream tensor types. Both come from the patch in this repo.
2. **`-fa off` is required for the vision path** on gfx1151. Text-only serving runs fine with
   `-fa on`.
3. **Images below 28×28 px collapse to a single merge token.** The preprocessor snaps to
   `patch 14 × merge 2`, so an 8×8 input carries no usable spatial signal and the model reports the
   dominant colour of the padded canvas. Feed images at 256×256 or larger.
4. **Vulkan / CUDA / CPU cannot load these files** — ROCmFP4/ROCmFPX are ROCm-only formats.

## Independent results

None yet. If you run this build, please open a discussion with hardware, GPU, ROCm version, runtime
commit, exact command, context, prompt-processing tok/s, generation tok/s and peak RAM. Independent
reproductions will be listed separately from author benchmarks and carry more weight.

## License and attribution

Base model and its licence are the Muse team's. ROCmFP4 / ROCmFPX quantisation types are from the
ROCmFPX fork of llama.cpp. This repository contains the quantised weights, the architecture port and
the measurements above.

<!-- CREDITS:START -->

## Acknowledgements

**[ROCmFPX](https://github.com/charlie12345/ROCmFPX) — maintained by
[`charlie12345`](https://github.com/charlie12345)**
The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every file here
was produced with its `llama-quantize` and runs on its runtime. Licensed MIT, based on upstream
llama.cpp.

**[llama.cpp](https://github.com/ggml-org/llama.cpp) — ggml-org and contributors**
The inference engine, GGUF format and conversion tooling everything here is built on.

**[AMD ROCm](https://github.com/ROCm/ROCm)** — the compute platform these builds target, on
gfx1151 / Radeon 8060S.

**[Muse](https://huggingface.co/meta-models)** — the base model and its licence are theirs. This
repository contributes the architecture port, quantisation and measurement only.

If you use these files, please credit ROCmFPX alongside this repository.

<!-- CREDITS:END -->