Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,58 @@
+---
+license: mit
+tags: [qhexrt, hexagon, npu, qnn, llm, deepseek-r1, qwen2]
+base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
+pipeline_tag: text-generation
+---
+
+# DeepSeek-R1-Distill-Qwen-1.5B — Hexagon NPU (QHexRT) bundle
+
+A prebuilt **QHexRT** bundle that runs **DeepSeek-R1-Distill-Qwen-1.5B** fully on the **Qualcomm Hexagon NPU**
+(no CPU/GPU in the compute loop). Drop the arch dir on a device and run with `qhx_generate`.
+
+- **Arch present:** `v81/` — **Hexagon v81** (Snapdragon 8 Elite Gen 2 class, SM8850, `soc_model 87`).
+- **Arch-pinned:** a context binary won't load on another Hexagon arch (the `soc_model`+`dsp_arch` are baked
+  in). Each arch is its own flat `<arch>/` dir.
+
+## What's optimized
+- **GQA-native decode** (12 query / 2 KV heads grouped, no `repeat_interleave`), in-graph attention with a
+  fixed-MAXCTX (1024) KV cache + 1D causal mask, **NPU lm-head**, host embed lookup + RoPE (θ=1e4) + sampler.
+- **fp16 throughout** (decode/prefill/lm-head). W8 weight-only quant is available but **degrades this small
+  model** (first token correct, continuation drifts) — fp16 is the validated greedy-exact bundle.
+- **v81 f16-attention-overflow fix (critical for this family):** Qwen2/2.5 has **no QK-norm**, so q/k stay
+  unnormalized (|k| up to ~400) and the attention `q·k` products overflow fp16 (>65504) on the HTP → garbage.
+  The export folds the `1/sqrt(d)` scale **into q before** the matmul and computes the self-score as a
+  **matmul** (HMX f32-accumulate, not an fp16 reduce). Math-identical; makes decode **greedy-exact** on v81.
+
+## Files (`v81/`)
+| file | role |
+|---|---|
+| `DeepSeek-R1-Distill-Qwen-1.5B.json` | QHexRT manifest (the declarative run plan) |
+| `deepseekr1distillqwen15b_prefill_f16.bin` | prefill context (fp16) |
+| `deepseekr1distillqwen15b_decode_f16.bin` | decode context (fp16) |
+| `deepseekr1distillqwen15b_lmhead_f16.bin` | lm-head context (fp16) |
+| `deepseekr1distillqwen15b_embed_f16.bin` | input embedding table (fp16, host lookup) |
+| `tokenizer.json` | tokenizer |
+
+## Run
+```bash
+hf download runanywhere/deepseek_r1_distill_qwen_1_5b_HNPU --local-dir ds15b
+adb push ds15b/v81 /data/local/tmp/wq/ds15b           # PowerShell + native paths on Windows
+# QNN runtime libs (libQnnHtp.so/libQnnSystem.so + v81 HTP skel) come from the QAIRT SDK, not this repo
+adb shell "cd /data/local/tmp/wq && LD_LIBRARY_PATH=. \
+  ./qhx_generate ds15b/DeepSeek-R1-Distill-Qwen-1.5B.json libQnnHtp.so libQnnSystem.so ds15b \
+  64 'The capital of France is'"
+```
+Tool arg order is invariant: `<tool> <manifest> libQnnHtp.so libQnnSystem.so <artifacts_root> …`.
+
+## Measured (v81 / SM8850, qhx_generate)
+- **Parity:** **greedy-exact** vs the HF fp32 reference — 10/10 gold tokens
+  (`"The capital of France is"` → `" Paris, and the capital of Germany is Berlin."`), and generalizes
+  (e.g. `"What is 15 multiplied by 6?"` → coherent step-by-step reasoning).
+- **Decode:** ~13.5 tok/s (73.9 ms/tok), MAXCTX 1024.
+
+## Caveats
+- **fp16 bundle** (validated). W8 weight-only was tried but degrades this small model (W4 is
+  HTP-toolchain-blocked); fp16 is shipped for correctness.
+- Decode-over-prompt seeding (no batched-prefill path at runtime). **v81-pinned** — won't load on other
+  Hexagon arches; re-finalize per arch.