---
license: mit
tags: [qhexrt, hexagon, npu, qnn, llm, deepseek-r1, qwen2]
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
pipeline_tag: text-generation
---

# DeepSeek-R1-Distill-Qwen-1.5B — Hexagon NPU (QHexRT) bundle

A prebuilt **QHexRT** bundle that runs **DeepSeek-R1-Distill-Qwen-1.5B** fully on the **Qualcomm Hexagon NPU**
(no CPU/GPU in the compute loop). Drop the arch dir on a device and run with `qhx_generate`.

- **Arch present:** `v81/` — **Hexagon v81** (Snapdragon 8 Elite Gen 2 class, SM8850, `soc_model 87`).
- **Arch-pinned:** a context binary won't load on another Hexagon arch (the `soc_model`+`dsp_arch` are baked
  in). Each arch is its own flat `<arch>/` dir.

## What's optimized
- **GQA-native decode** (12 query / 2 KV heads grouped, no `repeat_interleave`), in-graph attention with a
  fixed-MAXCTX (1024) KV cache + 1D causal mask, **NPU lm-head**, host embed lookup + RoPE (θ=1e4) + sampler.
- **fp16 throughout** (decode/prefill/lm-head). W8 weight-only quant is available but **degrades this small
  model** (first token correct, continuation drifts) — fp16 is the validated greedy-exact bundle.
- **v81 f16-attention-overflow fix (critical for this family):** Qwen2/2.5 has **no QK-norm**, so q/k stay
  unnormalized (|k| up to ~400) and the attention `q·k` products overflow fp16 (>65504) on the HTP → garbage.
  The export folds the `1/sqrt(d)` scale **into q before** the matmul and computes the self-score as a
  **matmul** (HMX f32-accumulate, not an fp16 reduce). Math-identical; makes decode **greedy-exact** on v81.

## Files (`v81/`)
| file | role |
|---|---|
| `DeepSeek-R1-Distill-Qwen-1.5B.json` | QHexRT manifest (the declarative run plan) |
| `deepseekr1distillqwen15b_prefill_f16.bin` | prefill context (fp16) |
| `deepseekr1distillqwen15b_decode_f16.bin` | decode context (fp16) |
| `deepseekr1distillqwen15b_lmhead_f16.bin` | lm-head context (fp16) |
| `deepseekr1distillqwen15b_embed_f16.bin` | input embedding table (fp16, host lookup) |
| `tokenizer.json` | tokenizer |

## Run
```bash
hf download runanywhere/deepseek_r1_distill_qwen_1_5b_HNPU --local-dir ds15b
adb push ds15b/v81 /data/local/tmp/wq/ds15b           # PowerShell + native paths on Windows
# QNN runtime libs (libQnnHtp.so/libQnnSystem.so + v81 HTP skel) come from the QAIRT SDK, not this repo
adb shell "cd /data/local/tmp/wq && LD_LIBRARY_PATH=. \
  ./qhx_generate ds15b/DeepSeek-R1-Distill-Qwen-1.5B.json libQnnHtp.so libQnnSystem.so ds15b \
  64 'The capital of France is'"
```
Tool arg order is invariant: `<tool> <manifest> libQnnHtp.so libQnnSystem.so <artifacts_root> …`.

## Measured (v81 / SM8850, qhx_generate)
- **Parity:** **greedy-exact** vs the HF fp32 reference — 10/10 gold tokens
  (`"The capital of France is"` → `" Paris, and the capital of Germany is Berlin."`), and generalizes
  (e.g. `"What is 15 multiplied by 6?"` → coherent step-by-step reasoning).
- **Decode:** ~13.5 tok/s (73.9 ms/tok), MAXCTX 1024.

## Caveats
- **fp16 bundle** (validated). W8 weight-only was tried but degrades this small model (W4 is
  HTP-toolchain-blocked); fp16 is shipped for correctness.
- Decode-over-prompt seeding (no batched-prefill path at runtime). **v81-pinned** — won't load on other
  Hexagon arches; re-finalize per arch.