---
license: apache-2.0
tags:
  - qhexrt
  - hexagon
  - npu
  - qnn
  - llm
language:
  - en
base_model: Qwen/Qwen3-0.6B
pipeline_tag: text-generation
---

# Qwen3-0.6B — QHexRT NPU bundle (Hexagon v81)

Precompiled **Qwen3-0.6B** LLM for the **QHexRT** runtime on the Qualcomm Hexagon **v81** NPU
(Snapdragon 8 Elite Gen 5 / **SM8850**, soc_model 87). A first-class **native** QHexRT LLM (no Genie):
host embed lookup → NPU decode (GQA-native, 16 query heads) → NPU tied lm-head, driven by the
`qwen3_generate` host-op. Compiled with the **forge** pipeline (oracle-gated export → QAIRT 2.47 compile).

**Arch-pinned:** a v81 context binary will not load on another Hexagon arch — re-target by re-running the
recipe (`recipes/qwen3-0.6b/convert.sh <arch>`).

## What's optimized
- **GQA-native decode** (16 q-heads ≤ 16 → grouped K/V, no `repeat_interleave`, no VTCM spill), full-dim 1D
  RoPE (θ=1e6), QK-norm, sliding-window KV ring, decode-over-prompt seed + batched prefill.
- **NPU tied lm-head** (`x @ Eᵀ` on HMX), host embed lookup, host RoPE/cmask.
- **W8A16** weight-only decode + lmhead (the v81 quant floor; W4 is HTP-toolchain-blocked), **f16** prefill,
  compiled at **O3 / vtcm 8 MB**.

## Files (`v81/`)
| file | role |
|------|------|
| `qwen3-0.6b-1024final.json` | QHexRT manifest (LLM family, `qwen3_generate` plan, MAXCTX 1024) |
| `qwen06_pf_f16.bin` | prefill context binary (f16, batched prompt ingest) |
| `qwen06_decn1k_w8.bin` | decode context binary (W8A16, MAXCTX 1024) |
| `qwen06_lmh_w8.bin` | tied lm-head context binary (W8, `x@Eᵀ` on HMX) |
| `qwen06_embed_f16.bin` | embedding table (host lookup) |
| `tokenizer.json` | Qwen2 BPE tokenizer (vocab 151936) |

## Run (QHexRT CLI)
```bash
huggingface-cli download runanywhere/qwen3_0_6b_HNPU --local-dir qwen3_0_6b_HNPU
# QNN libs come from the QAIRT SDK (lib/aarch64-android) + the v81 HTP skel; push them next to qhx_generate.
adb push qwen3_0_6b_HNPU/v81 /data/local/tmp/wq/qwen06
adb shell "cd /data/local/tmp/wq && LD_LIBRARY_PATH=. \
  ./qhx_generate qwen06/qwen3-0.6b-1024final.json libQnnHtp.so libQnnSystem.so qwen06 64 'The capital of France is'"
# -> " Paris. The capital of Italy is Rome. The capital of Spain is Madrid. ..."
```

## Measured (SM8850 / v81, soc 87, QAIRT 2.47)
- **Parity:** greedy-**exact** vs HF `Qwen/Qwen3-0.6B` — the canonical prompt matches **64/64** tokens.
- **Speed:** **38.7 tok/s** decode (24 ms/tok: 20 graph + 4 lm-head), **43.1 tok/s** prefill, TTFT ≈ 116 ms.
- **Context-length sweep** (greedy, to 70 % MAXCTX): coherent through **512 tokens**; the W8 decode drifts to
  empty output at ~717 (70 %). Treat **512 (50 % MAXCTX)** as the reliable W8 context on v81.

## Caveats — W8 acceptance (read before relying on strict parity)
The **fp16 export gate is 8/8** (the math is correct), but **W8 decode does not hold strict greedy-exact
across a diverse prompt suite** — only 2/6 suite prompts match HF token-for-token (W8 argmax drift, not a
bug). This bundle is the **size/speed-optimized best-effort** artifact: its single canonical prompt is
greedy-exact, but arbitrary prompts may diverge from HF after a few tokens while staying coherent. For
**guaranteed strict parity**, compile an **fp16** decode/lm-head bundle instead (`recipes/qwen3-0.6b`, set the
decode/lmhead graphs to `precision: f16`). This mirrors the QHexRT W8 acceptance policy.

_Built + device-validated with [QHexRT](https://github.com/RunanywhereAI/QHexRT) `forge` — `recipes/qwen3-0.6b`._