Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,61 @@
+---
+license: other
+license_name: lfm1.0
+license_link: https://huggingface.co/LiquidAI/LFM2.5-230M/blob/main/LICENSE
+base_model: LiquidAI/LFM2.5-230M
+tags:
+  - qualcomm
+  - hexagon
+  - npu
+  - qnn
+  - qhexrt
+  - on-device
+  - lfm2
+language:
+  - en
+pipeline_tag: text-generation
+---
+
+# LFM2.5-230M — Hexagon NPU (QHexRT) bundle
+
+[LiquidAI/LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M) compiled to run on the **Qualcomm Hexagon
+v79 NPU** (Snapdragon 8 Elite / SM8750, e.g. Galaxy S25) via the **[QHexRT](https://github.com/RunanywhereAI/QHexRT)**
+runtime. Pure on-device inference — **no Python in the hot path.**
+
+14-layer hybrid model (6 GQA-attention + 8 short-conv), hidden 1024, vocab 65536, tied lm-head. Runs **W8
+weight-only** (int8 weights, fp16 activations) with **GQA-native decode** + **batched prefill** + an
+**on-NPU lm-head**. Greedy output matches the HF model exactly (`"The capital of France is"` → `" Paris."`).
+
+## Measured (Samsung S25, Hexagon v79)
+
+| config | decode | prefill | peak RAM |
+|---|---|---|---|
+| **MAXCTX 512** (`lfm2-5-230m.json`) | **164 tok/s** (6.1 ms/tok) | batched, ~17k tok/s (≤512 prompt) | ~430 MB |
+| **MAXCTX 2048** (`lfm2-5-230m-2048.json`) | **127 tok/s** (7.9 ms/tok) | **~8,950 tok/s** (2k-token prompt in 227 ms) | ~440 MB |
+
+For reference, LiquidAI's published **S25 CPU (int4)** numbers on 2k input are **prefill 1158 / decode 213
+tok/s** — this NPU bundle does prefill **~7.7× faster**, at far lower power. (Decode is W8 here: 4-bit weights
+are blocked by the v79 HTP toolchain, so W8 is the floor; the NPU still wins prefill and frees the CPU.)
+
+## Contents (`v79/`)
+
+| file | what |
+|---|---|
+| `lfm230_dec_512_w8.bin` / `lfm230_dec_2048_w8.bin` | W8 GQA-native decode (MAXCTX 512 / 2048) |
+| `lfm230_pf_512_w8.bin` / `lfm230_pf_2048_w8.bin` | W8 batched prefill (PN 512 / 2048) |
+| `lfm230_lmh_w8.bin` | W8 tied lm-head `hidden[1,1024]→logits[1,65536]` on-NPU |
+| `lfm_embed_f16.bin` | tied embedding table (host token→hidden lookup) |
+| `tokenizer.json` | the LFM2.5 tokenizer |
+| `lfm2-5-230m.json` / `lfm2-5-230m-2048.json` | QHexRT manifests (512 / 2048; declare the 14-layer `attn_idx`/`conv_idx` schedule) |
+
+## Run
+
+```bash
+hf download runanywhere/lfm2_5_230m_HNPU --local-dir lfm2_5_230m_HNPU
+adb push lfm2_5_230m_HNPU/v79 /data/local/tmp/lfm230     # PowerShell + native paths on Windows
+adb shell "cd /data/local/tmp/lfm230 && LD_LIBRARY_PATH=. \
+  ./qhx_generate lfm2-5-230m-2048.json libQnnHtp.so libQnnSystem.so . 64 'The capital of France is'"
+```
+
+(One-time: stage the QAIRT v79 runtime libs + the `qhx_generate` tool into the same dir — see the QHexRT
+deploy docs.) Arch-pinned to **v79**; a v79 binary will not load on other Hexagon arches.