Add v81 (SM8850) W8 bundle + CPU head-to-head to model card (#1)

- Add v81 (SM8850) W8 bundle + CPU head-to-head to model card (dbfa1f2ac936584915cff952f46ba95ce4f4458d)
- Model card: v81-only, NPU-advantage metrics, concise (84a1a09ebc7a4ea8c2fd90168eefc8e2a8529658)


Co-authored-by: Sanchit Monga <sanmonga22@users.noreply.huggingface.co>

README.md

@@ -16,46 +16,37 @@ language:
 pipeline_tag: text-generation
 ---
 
-# LFM2.5-230M — Hexagon NPU (QHexRT) bundle
+# LFM2.5-230M — Hexagon v81 NPU (QHexRT)
 
-[LiquidAI/LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M) compiled to run on the **Qualcomm Hexagon
-v79 NPU** (Snapdragon 8 Elite / SM8750, e.g. Galaxy S25) via the **[QHexRT](https://github.com/RunanywhereAI/QHexRT)**
-runtime. Pure on-device inference — **no Python in the hot path.**
+[LiquidAI/LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M) running fully on the **Qualcomm Hexagon
+v81 NPU** (Snapdragon 8 Elite Gen-2 / SM8850) via the **[QHexRT](https://github.com/RunanywhereAI/QHexRT)**
+runtime — **no Python in the hot path**. W8 weight-only, GQA-native decode, batched prefill, on-NPU lm-head.
+Greedy output is identical to the source model (`"The capital of France is"` → `" Paris."`).
 
-14-layer hybrid model (6 GQA-attention + 8 short-conv), hidden 1024, vocab 65536, tied lm-head. Runs **W8
-weight-only** (int8 weights, fp16 activations) with **GQA-native decode** + **batched prefill** + an
-**on-NPU lm-head**. Greedy output matches the HF model exactly (`"The capital of France is"` → `" Paris."`).
+## Why the NPU — measured on SM8850 (vs llama.cpp CPU, same device)
 
-## Measured (Samsung S25, Hexagon v79)
-
-| config | decode | prefill | peak RAM |
+| metric | Hexagon v81 NPU | CPU (llama.cpp Q8_0) | NPU advantage |
 |---|---|---|---|
-| **MAXCTX 512** (`lfm2-5-230m.json`) | **164 tok/s** (6.1 ms/tok) | batched, ~17k tok/s (≤512 prompt) | ~430 MB |
-| **MAXCTX 2048** (`lfm2-5-230m-2048.json`) | **127 tok/s** (7.9 ms/tok) | **~8,950 tok/s** (2k-token prompt in 227 ms) | ~440 MB |
-
-For reference, LiquidAI's published **S25 CPU (int4)** numbers on 2k input are **prefill 1158 / decode 213
-tok/s** — this NPU bundle does prefill **~7.7× faster**, at far lower power. (Decode is W8 here: 4-bit weights
-are blocked by the v79 HTP toolchain, so W8 is the floor; the NPU still wins prefill and frees the CPU.)
+| **Prefill** | **12,540 tok/s** | 871 tok/s | **~14× faster** |
+| **Time-to-first-token** (512-token prompt) | **~36 ms** (flat) | 588 ms | **~16× lower** |
+| **End-to-end** (512-token prompt + 128 new) | **0.77 s** | 1.13 s | **~1.5× faster** |
 
-## Contents (`v79/`)
-
-| file | what |
-|---|---|
-| `lfm230_dec_512_w8.bin` / `lfm230_dec_2048_w8.bin` | W8 GQA-native decode (MAXCTX 512 / 2048) |
-| `lfm230_pf_512_w8.bin` / `lfm230_pf_2048_w8.bin` | W8 batched prefill (PN 512 / 2048) |
-| `lfm230_lmh_w8.bin` | W8 tied lm-head `hidden[1,1024]→logits[1,65536]` on-NPU |
-| `lfm_embed_f16.bin` | tied embedding table (host token→hidden lookup) |
-| `tokenizer.json` | the LFM2.5 tokenizer |
-| `lfm2-5-230m.json` / `lfm2-5-230m-2048.json` | QHexRT manifests (512 / 2048; declare the 14-layer `attn_idx`/`conv_idx` schedule) |
+Batched **O(1) prefill** holds TTFT flat at **~36 ms regardless of prompt length**, so the NPU pulls further
+ahead the longer the context — at far lower power than driving 8 CPU cores at max clock.
 
 ## Run
 
 ```bash
 hf download runanywhere/lfm2_5_230m_HNPU --local-dir lfm2_5_230m_HNPU
-adb push lfm2_5_230m_HNPU/v79 /data/local/tmp/lfm230     # PowerShell + native paths on Windows
+adb push lfm2_5_230m_HNPU/v81 /data/local/tmp/lfm230     # PowerShell + native paths on Windows
 adb shell "cd /data/local/tmp/lfm230 && LD_LIBRARY_PATH=. \
-  ./qhx_generate lfm2-5-230m-2048.json libQnnHtp.so libQnnSystem.so . 64 'The capital of France is'"
+  ./qhx_generate lfm2-5-230m.json libQnnHtp.so libQnnSystem.so . 64 'The capital of France is'"
 ```
 
-(One-time: stage the QAIRT v79 runtime libs + the `qhx_generate` tool into the same dir — see the QHexRT
-deploy docs.) Arch-pinned to **v79**; a v79 binary will not load on other Hexagon arches.
+Stage the QAIRT v81 runtime libs (`libQnnHtp.so`, `libQnnSystem.so`, `libQnnHtpV81Skel.so`/`Stub.so`) + the
+`qhx_generate` tool into the same dir (from the QAIRT SDK; see the
+[QHexRT deploy docs](https://github.com/RunanywhereAI/QHexRT)). Context binaries are **arch-pinned to v81**.
+
+## `v81/`
+`lfm2-5-230m.json` (manifest) · `lfm230_dec_512_w8.bin` (decode) · `lfm230_pf_512_w8.bin` (prefill) ·
+`lfm230_lmh_w8.bin` (lm-head) · `lfm_embed_f16.bin` (embeddings) · `tokenizer.json`

v81/lfm2-5-230m.json

@@ -0,0 +1,51 @@
+{
+  "schema_version": 1,
+  "_comment": "LFM2.5-230M @ MAXCTX=512, W8 weight-only (int8 weights, fp16 activations) + GQA-native decode + batched prefill (PN=512) + NPU lm-head. 14-layer hybrid = 8 short-conv + 6 GQA-attention; per-layer schedule via attn_idx/conv_idx (the op defaults to the 16-layer 350M when absent).",
+  "model": {
+    "name": "lfm2-5-230m",
+    "family": "llm",
+    "dsp_arch": "v81"
+  },
+  "params": {
+    "hidden": 1024,
+    "vocab": 65536,
+    "n_layers": 14,
+    "max_ctx": 512,
+    "kv_dim": 512,
+    "head_dim": 64,
+    "rope_theta": 1000000.0,
+    "eos_token_id": 7
+  },
+  "artifacts": {
+    "contexts": {
+      "decode": {
+        "bin": "lfm230_dec_512_w8.bin"
+      },
+      "prefill": {
+        "bin": "lfm230_pf_512_w8.bin"
+      },
+      "lmhead": {
+        "bin": "lfm230_lmh_w8.bin"
+      }
+    },
+    "embed": "lfm_embed_f16.bin",
+    "tokenizer": "tokenizer.json"
+  },
+  "plan": {
+    "steps": [
+      {
+        "host": "lfm_generate",
+        "params": {
+          "prefill": "lfm230_pf_512_w8",
+          "decode": "lfm230_dec_512_w8",
+          "lmhead": "lfm230_lmh_w8",
+          "attn_idx": "2,4,6,8,10,12",
+          "conv_idx": "0,1,3,5,7,9,11,13"
+        }
+      },
+      {
+        "emit": "all"
+      }
+    ]
+  }
+}

v81/lfm230_dec_512_w8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a357684ca14aaa9876211667a5d1d11e460bb159a35c9f2bf47a6d56fdf45e4d
+size 164409344

v81/lfm230_lmh_w8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35cc61ae2772445e77751d4496dbcb135c31dab069529e17174630270563c50c
+size 68313088

v81/lfm230_pf_512_w8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f71af58e350dd0d8ada68c75c80ff7b24de4cadd6f17026ebd6b5d3cb309556
+size 167096320

v81/lfm_embed_f16.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19f7edcbb7399d6ac2f75d532d08b7fe4f2784a2f4da750c021561af05be0b1b
+size 134217728