sm280299 commited on
Commit
1ec3e37
·
verified ·
1 Parent(s): cadbdbd

QHexRT launch: 100% NPU, multimodal roadmap, remove open-source refs

Browse files
Files changed (1) hide show
  1. README.md +10 -6
README.md CHANGED
@@ -19,9 +19,13 @@ pipeline_tag: text-generation
19
  # LFM2.5-230M — Hexagon v81 NPU (QHexRT)
20
 
21
  [LiquidAI/LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M) running fully on the **Qualcomm Hexagon
22
- v81 NPU** (Snapdragon 8 Elite Gen-2 / SM8850) via the **[QHexRT](https://github.com/RunanywhereAI/QHexRT)**
23
- runtime **no Python in the hot path**. W8 weight-only, GQA-native decode, batched prefill, on-NPU lm-head.
24
- Greedy output is identical to the source model (`"The capital of France is"` → `" Paris."`).
 
 
 
 
25
 
26
  ## Why the NPU — measured on SM8850 (vs llama.cpp CPU, same device)
27
 
@@ -34,7 +38,7 @@ Greedy output is identical to the source model (`"The capital of France is"` →
34
  Batched **O(1) prefill** holds TTFT flat at **~36 ms regardless of prompt length**, so the NPU pulls further
35
  ahead the longer the context — at far lower power than driving 8 CPU cores at max clock.
36
 
37
- **Full benchmark write-up:** [runanywhere.ai/blog](https://www.runanywhere.ai/blog/qhexrt-npu-vs-cpu-lfm2.5-230m-qualcomm-hexagon) · [article draft in this repo](blog/qhexrt-npu-vs-cpu-lfm2.5-230m.md)
38
 
39
  ## Run
40
 
@@ -46,8 +50,8 @@ adb shell "cd /data/local/tmp/lfm230 && LD_LIBRARY_PATH=. \
46
  ```
47
 
48
  Stage the QAIRT v81 runtime libs (`libQnnHtp.so`, `libQnnSystem.so`, `libQnnHtpV81Skel.so`/`Stub.so`) + the
49
- `qhx_generate` tool into the same dir (from the QAIRT SDK; see the
50
- [QHexRT deploy docs](https://github.com/RunanywhereAI/QHexRT)). Context binaries are **arch-pinned to v81**.
51
 
52
  ## `v81/`
53
  `lfm2-5-230m.json` (manifest) · `lfm230_dec_512_w8.bin` (decode) · `lfm230_pf_512_w8.bin` (prefill) ·
 
19
  # LFM2.5-230M — Hexagon v81 NPU (QHexRT)
20
 
21
  [LiquidAI/LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M) running fully on the **Qualcomm Hexagon
22
+ v81 NPU** (Snapdragon 8 Elite Gen-2 / SM8850) via **QHexRT** — RunAnywhere's inference engine for
23
+ Qualcomm NPUs. **100% on the HTP.** No Python in the hot path. W8 weight-only, GQA-native decode,
24
+ batched prefill, on-NPU lm-head. Greedy output is identical to the source model
25
+ (`"The capital of France is"` → `" Paris."`).
26
+
27
+ QHexRT is the first engine built to run **LLM, VLM, STT, TTS, and embeddings** fully on Qualcomm
28
+ Hexagon NPUs. LFM 2.5 230M is the first model in the catalog.
29
 
30
  ## Why the NPU — measured on SM8850 (vs llama.cpp CPU, same device)
31
 
 
38
  Batched **O(1) prefill** holds TTFT flat at **~36 ms regardless of prompt length**, so the NPU pulls further
39
  ahead the longer the context — at far lower power than driving 8 CPU cores at max clock.
40
 
41
+ **Full launch write-up:** [runanywhere.ai/blog](https://www.runanywhere.ai/blog/qhexrt-npu-vs-cpu-lfm2.5-230m-qualcomm-hexagon) · [article draft in this repo](blog/qhexrt-npu-vs-cpu-lfm2.5-230m.md)
42
 
43
  ## Run
44
 
 
50
  ```
51
 
52
  Stage the QAIRT v81 runtime libs (`libQnnHtp.so`, `libQnnSystem.so`, `libQnnHtpV81Skel.so`/`Stub.so`) + the
53
+ `qhx_generate` tool into the same dir (from the QAIRT SDK). Context binaries are **arch-pinned to v81**.
54
+ Contact [san@runanywhere.ai](mailto:san@runanywhere.ai) for QHexRT deployment access.
55
 
56
  ## `v81/`
57
  `lfm2-5-230m.json` (manifest) · `lfm230_dec_512_w8.bin` (decode) · `lfm230_pf_512_w8.bin` (prefill) ·