---
license: apache-2.0
base_model: AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16
library_name: gguf
tags:
- gguf
- qwen3
- qwen3.6
- aeon
- rocm
- rocmfp4
- rocmfp6
- rocmfpx
- strix-halo
- llama.cpp
- speculative-decoding
- mtp
- amd
datasets: []
---

![AEON Ultimate ROCmFP4/ROCmFP6 Strix Halo benchmark banner](./aeon-rocmfpx-strix-halo-banner.jpg)

# AEON Ultimate ROCmFP4/ROCmFP6 GGUF for AMD Strix Halo

This repo contains AMD Strix Halo GGUF quantizations of
[AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16](https://huggingface.co/AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16),
including the bundled Qwen3.6 MTP tensors for llama.cpp speculative decoding.

These files require custom Strix Halo runtimes. They are not for stock llama.cpp and not for vLLM.

- ROCmFP4 runtime: [charlie12345/rocmfp4-llama](https://github.com/charlie12345/rocmfp4-llama), branch `mtp-rocmfp4-strix`.
- ROCmFP6 runtime: [charlie12345/ROCmFPX](https://github.com/charlie12345/ROCmFPX), tested at commit `11d76c2`.

## Files

| File | Format | Use | Size | SHA256 |
|---|---|---|---:|---|
| `AEON-Ultimate-ROCmFP4-STRIX_LEAN-MTP.gguf` | ROCmFP4 STRIX_LEAN | Recommended FP4 speed/default build | 14.8 GiB | `9afd704c1fc85789c86f9b8099aa736952af13967b10a3ad11f6c2306aeaa9e2` |
| `AEON-Ultimate-ROCmFP4-STRIX-MTP.gguf` | ROCmFP4 STRIX | Quality-biased FP4 preset | 15.0 GiB | `9dc52099a62ce4cdb113e6383eed0efa5b726ed928c21bc9caa107f2ce9a8cb2` |
| `AEON-Ultimate-ROCmFP6-MTP.gguf-00001-of-00006.gguf` through `00006-of-00006` | `Q6_0_ROCMFPX` split GGUF | Recommended FP6 speed/default build; load the first shard | 6 shards, 20.97 GiB total | See `SHA256SUMS` |
| `AEON-Ultimate-ROCmFP6-AGENT-MTP.gguf-00001-of-00007.gguf` through `00007-of-00007` | `Q6_0_ROCMFPX_AGENT` split GGUF | Quality-biased FP6 profile for coding harnesses and tool-call style workloads; load the first shard | 7 shards, 23.54 GiB total | See `SHA256SUMS` |

All variants were quantized from the BF16 source GGUF with the MTP extension bundled. Do not requantize from the FP4 files.

## Which File Should I Use?

For lowest latency on one Strix Halo, start with `AEON-Ultimate-ROCmFP4-STRIX_LEAN-MTP.gguf`. It had the best measured MTP decode speed in my coding-agent tests.

For a higher-quality FP6 experiment, start with `AEON-Ultimate-ROCmFP6-AGENT-MTP.gguf-00001-of-00007.gguf`. It preserves more tensors at Q8 under ROCmFPX's agent routing and is the most plausible coding-harness candidate among the FP6 files. It is slower than default FP6 in a short decode smoke test.

The FP6 variants are uploaded as split GGUFs with shards under 4 GB to avoid large multipart-upload issues. Put all shards for a variant in the same directory and load the `00001-of-...` file; llama.cpp will find the remaining shards.

For parallel subagent pools, test no speculative decoding as well as MTP. In my FP4 tests, MTP helped single-stream latency but did not scale well across four parallel slots.

## Build the ROCmFP4 Runtime

```bash
git clone https://github.com/charlie12345/rocmfp4-llama.git
cd rocmfp4-llama
git checkout mtp-rocmfp4-strix
env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh
```

## Build the ROCmFP6 Runtime

```bash
git clone https://github.com/charlie12345/ROCmFPX.git
cd ROCmFPX
git checkout 11d76c2
env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh llama-cli llama-server llama-bench
```

The build directory is still named `build-strix-rocmfp4` because ROCmFPX extends the same Strix Halo llama.cpp fork.

Common Strix Halo environment:

```bash
export HSA_OVERRIDE_GFX_VERSION=11.5.1
export GGML_HIP_ENABLE_UNIFIED_MEMORY=1
export LD_LIBRARY_PATH=$PWD/build-strix-rocmfp4/bin:/opt/rocm/lib:${LD_LIBRARY_PATH:-}
```

## Recommended FP6 Coding Server

ROCmFPX includes a helper that auto-checks MTP metadata and starts llama-server with conservative `draft-mtp` settings:

```bash
cd ROCmFPX
MODEL=/path/to/AEON-Ultimate-ROCmFP6-AGENT-MTP.gguf-00001-of-00007.gguf \
DEVICE=ROCm0 \
SPEC_DRAFT_DEVICE=ROCm0 \
CTX_SIZE=32768 \
PARALLEL=1 \
SPEC_DRAFT_N_MAX=4 \
PERF_PRESET=latency \
scripts/run-rocmfpx-mtp-server.sh
```

The helper runs with `--reasoning off`, `--reasoning-format none`, Jinja templates, full offload, flash attention, F16 KV cache, and `draft-mtp` enabled. That matters for OpenAI-compatible coding harnesses: with reasoning enabled, Qwen-style reasoning can be routed to `reasoning_content` while normal `content` appears empty to clients.

Equivalent core server args:

```bash
./build-strix-rocmfp4/bin/llama-server \
  -m /path/to/AEON-Ultimate-ROCmFP6-AGENT-MTP.gguf-00001-of-00007.gguf \
  --alias aeon-fp6-agent --host 127.0.0.1 --port 8080 \
  --jinja --reasoning off --reasoning-format none \
  -c 32768 -ngl 999 -fa on -dev ROCm0 \
  -b 2048 -ub 512 -t 16 -tb 32 \
  -ctk f16 -ctv f16 \
  --spec-type draft-mtp \
  --spec-draft-device ROCm0 --spec-draft-ngl all \
  --spec-draft-type-k f16 --spec-draft-type-v f16 \
  --spec-draft-n-max 4 --spec-draft-n-min 0 \
  --spec-draft-p-min 0.0 --spec-draft-p-split 0.10 \
  --parallel 1 --metrics --no-webui
```

## Recommended FP4 Single-Stream Coding Server

This was the fastest tested single-stream coding-agent config:

```bash
./build-strix-rocmfp4/bin/llama-server \
  -m /path/to/AEON-Ultimate-ROCmFP4-STRIX_LEAN-MTP.gguf \
  --alias aeon --host 127.0.0.1 --port 8080 \
  --jinja -rea off \
  -c 32768 -ngl 999 -fa on -dev ROCm0 \
  -b 512 -ub 512 -t 16 -tb 32 \
  -ctk f16 -ctv f16 \
  --spec-type draft-mtp \
  --spec-draft-device ROCm0 --spec-draft-ngl all \
  --spec-draft-type-k f16 --spec-draft-type-v f16 \
  --spec-draft-n-max 4 --spec-draft-n-min 0 --spec-draft-p-min 0.0 \
  --parallel 1 --metrics --no-mmap
```

## Recommended Parallel Subagent Server

For a second Strix Halo dedicated to several lower-priority subagents, test no speculative decoding with four parallel slots:

```bash
./build-strix-rocmfp4/bin/llama-server \
  -m /path/to/AEON-Ultimate-ROCmFP4-STRIX_LEAN-MTP.gguf \
  --alias aeon --host 127.0.0.1 --port 8081 \
  --jinja -rea off \
  -c 32768 -ngl 999 -fa on -dev ROCm0 \
  -b 512 -ub 512 -t 16 -tb 32 \
  -ctk f16 -ctv f16 \
  --parallel 4 --metrics --no-mmap
```

## Measured Strix Halo Performance

Hardware: Ryzen AI Max+ 395 / Radeon 8060S, gfx1151, 128 GB unified memory.

FP4 runtime: `rocmfp4-llama` branch `mtp-rocmfp4-strix`, commit `4795079b` plus a small local converter fix. Single cold coding prompt, 256 generated tokens:

| Config | Decode tok/s | Draft accepted |
|---|---:|---:|
| no speculative decoding | 13.95 | n/a |
| `draft-mtp`, n-max 2 | 30.24 | 164 / 182 |
| `draft-mtp`, n-max 3 | 36.10 | 185 / 209 |
| `draft-mtp`, n-max 4 | **38.46** | 195 / 238 |
| `draft-mtp`, n-max 5 | 38.10 | 201 / 266 |
| `draft-mtp`, n-max 6 | 36.80 | 205 / 295 |

Longer-context FP4 checks with the recommended single-stream config:

| Prompt tokens | Prefill tok/s | Decode tok/s | Generated |
|---:|---:|---:|---:|
| 6,357 | about 299-302 | 22.61 | 128 |
| 12,584 | 294.10 | 23.32 | 128 |
| 25,000 | 255.14 | 16.31 | 96 |

Parallel/subagent FP4 serving, varied coding prompts, 256 generated tokens per request:

| Mode | Parallel | Aggregate decode tok/s | Avg per-request decode tok/s |
|---|---:|---:|---:|
| MTP n-max 4 | 2 | 23.87 | 15.55 |
| MTP n-max 4 | 4 | 21.62 | 7.61 |
| no spec | 2 | 22.28 | 12.58 |
| no spec | 4 | **28.89** | 9.97 |

FP6 runtime smoke test: `ROCmFPX` commit `11d76c2`, `llama-bench -dev ROCm0 -ngl 999 -fa on -p 64 -n 64 -r 1`:

| Quant | Size | Prompt processing tok/s | Text generation tok/s |
|---|---:|---:|---:|
| `Q6_0_ROCMFPX` | 20.97 GiB | 57.82 | 9.44 |
| `Q6_0_ROCMFPX_AGENT` | 23.54 GiB | 57.66 | 8.65 |

These FP6 numbers are loader/throughput smoke tests, not full coding-agent MTP acceptance sweeps.

## STRIX vs STRIX_LEAN

`STRIX` is a quality-biased ROCmFP4 preset and `STRIX_LEAN` is a speed/size-biased preset. They are quantizations of the same AEON Ultimate weights.

On the tested Strix Halo, `STRIX` did not improve speed or MTP acceptance:

| Quant | Prompt | Decode tok/s | Draft accepted |
|---|---|---:|---:|
| STRIX_LEAN | short coding | **38.46** | 195 / 238 |
| STRIX | short coding | 38.28 | 195 / 238 |
| STRIX_LEAN | 8k-class prompt | **22.61** | 79 / 189 |
| STRIX | 8k-class prompt | 20.30 | 73 / 209 |

Use `STRIX_LEAN` unless your own correctness eval shows a quality win for `STRIX`.

## Provenance

- Source model: [AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16](https://huggingface.co/AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16)
- Base model: [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)
- FP4 runtime fork: [charlie12345/rocmfp4-llama](https://github.com/charlie12345/rocmfp4-llama)
- FP6 runtime fork: [charlie12345/ROCmFPX](https://github.com/charlie12345/ROCmFPX)
- BF16-to-GGUF conversion source revision: `d52a7bc77bf9c6c1bcff638bf20037e8dcb9af5b`
- ROCmFPX quantization revision: `11d76c2`

## License

Apache-2.0, inherited from the source model and Qwen/Qwen3.6-27B.