Qwen3.8-27B Quark checkpoints on RDNA4 (2x R9700, gfx1201, TP=2) — findings + asks

I run a 2x Radeon AI PRO R9700 (gfx1201) vLLM rig doing long-horizon agentic
evaluation (Terminal-Bench 2.1) on Qwen3.8-27B. I pulled both of your Quark
checkpoints this week, went through them and the vLLM Quark path in detail, and
found things I think are worth your time. Everything below is checkable in a
couple of minutes — file:line or PR number for each claim. Ranked by your effort,
cheapest first.

Short version

1. Neither checkpoint declares its MTP head as excluded

In both Qwen3.8-27B-Quark-AWQ-INT4-W4A16 and Qwen3.8-27B-Quark-AWQ-MXFP4:

• quantization_config.exclude is 112 entries = 111 model.visual.* blocks + lm_head
• layer_quant_config is {}
• but there are 15 mtp.* tensors in the safetensors, all BF16

So nothing tells vLLM the MTP head is unquantized. With
--speculative-config '{"method":"mtp","num_speculative_tokens":2}' this is the
AssertionError: self.data.shape == loaded_weight.shape that PR #52649 was
opened to fix — and that PR was closed on 2026-08-20 on the (correct) reasoning
that it belongs in Quark rather than vLLM, referencing internal #6109. Both
checkpoints predate that fix, so they're the ones still carrying it.

This is a config edit, and it matters disproportionately on Qwen3.5-family
models where the MTP head is the main decode-throughput lever.

Worth noting two community FP8 quantizers of the same base model already do this
correctly (AzatAI/Qwen3.8-27B-FP8-dynamic, ProCreations/grug-v1.1-qwen-3.8-27b-mtp-fp8),
so there's a reference for the expected shape of the exclusion list.

2. No KV-cache scales

kv_cache_quant_config is {} in both. vLLM already has the plumbing —
QuarkConfig.get_cache_scale_mapper() maps .k_proj.output_scale -> .attn.k_scale
— so a checkpoint that ships calibrated scales plugs straight into
--kv-cache-dtype fp8, which is a very common config on memory-constrained
cards. Right now everyone running fp8 KV on your checkpoints is falling back to
runtime scaling for free-ish quality they didn't have to give up.

3. The MXFP4 card should say it's emulated on RDNA4

This is the finding I most wanted to pass on, because it's invisible from the
model card and costs people a 20 GB download to discover.

vllm/platforms/rocm.py:

def supports_mx(cls) -> bool:
    return any(gfx in _GCN_ARCH for gfx in ["gfx95", "gfx1250"])

def supports_fp8(cls) -> bool:
    return on_cdna() or on_gfx12x()

gfx1201 passes supports_fp8() and fails supports_mx(). So on RDNA4 the MXFP4
checkpoint loads fine — the OCP-MX predicate matches, activations are
is_dynamic: true so nothing raises — and then runs the path that logs
"Simulated weight dequantization and activation QDQ ... linear layers computed
in high precision." QuarkW4A8_MXFP4_FP8 gates on the same
current_platform.supports_mx(), so the W4A8 variant lands in emulation too.

Net effect on RDNA4: the memory win is real (19.8 GB vs ~29 GB for FP8), but
there's no compute win, and you still pay the MXFP4 activation error through QDQ.
For a decode-bound workload that's likely a net regression against plain FP8.

One sentence on the card ("MX types are emulated on RDNA4/gfx12x; use FP8 there")
would save people the download. And separately — I'd genuinely like to know
whether supports_mx() excludes gfx12x because RDNA4 lacks the hardware, or
because the kernels aren't ported. I could only verify the software gate, and
the answer changes which format is worth targeting on this card.

4. The INT4 W4A16 checkpoint currently can't be loaded by any released vLLM

Qwen3.8-27B-Quark-AWQ-INT4-W4A16 is sitting at ~85k downloads. On vLLM
0.24.0+rocm723, quark/schemes/ contains nvfp4, ocp_mx, w4a8_mxfp4_fp8,
w8a8_fp8, w8a8_int8 — and _get_scheme_from_config has no int4/per-group branch,
so it falls through to
raise NotImplementedError("No quark compatible scheme was found").
The card points at #52642 for the W4A16Int4 scheme, and that PR is still open
with merge conflicts. So the required runtime doesn't exist in a release yet.

If you only do one expensive thing, I'd argue for this one, and specifically
for INT4-g128 over MXFP4 on RDNA4 — because it needs a dequant-and-WMMA GEMM
rather than MX hardware, and because the on-disk layout is already AWQ-shaped
([K, N/8] int32 + [K/128, N] bf16 scales), which is why #52642 is largely a
name remap plus pack-order canonicalization.

And the kernel work is already scoped in your own tree —
compressed_tensors_moe/rocm_moe_rdna.py:23:

# RDNA3 (gfx1100). Future: add RDNA4 (gfx12x), CDNA (gfx94x), etc.

For context on why this is worth it here: on this model 4-bit weights take the
checkpoint from ~29 GB to ~19.5 GB, which at TP=2 is ~4.75 GB freed per card. On
this architecture (16 full-attention layers of 64, 4 KV heads, head_dim 256,
~32 KB/token at fp8) that's roughly a doubling of usable KV pool. For long-horizon
agentic work KV headroom is the binding constraint, not weight bandwidth.

5. Minor: crc32.txt doesn't match either repo's own files

Both repos ship an identical 238-byte crc32.txt. On the INT4 repo, 4 of 8
entries disagree with the uploaded files (chat_template.jinja,
generation_config.json, preprocessor_config.json, tokenizer_config.json).
The commit history shows only README.md changed after the upload commit, so it
was published that way rather than going stale — looks like it was generated
against a different snapshot than what was uploaded.

Everything verifies clean against HF's own hashes (I checked sha256 on the
weights via the LFS oid and git-blob-sha1 on the rest, both repos, all files), so
this is cosmetic — but it makes a well-documented artifact look corrupted to
anyone who runs the check you shipped.

Happy to test any of this on real 2x RDNA4 hardware and report back with numbers
— I have the rig, the harness, and both checkpoints already staged. If a
gfx12x W4A16 kernel or a re-exported checkpoint would benefit from a second pair
of eyes on non-MI hardware, I'm glad to be that.