Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,80 @@
+---
+license: apache-2.0
+tags:
+  - qhexrt
+  - internvl
+  - vlm
+  - image-text-to-text
+  - hexagon
+  - npu
+  - qnn
+base_model: OpenGVLab/InternVL3_5-1B
+pipeline_tag: image-text-to-text
+language:
+  - en
+---
+
+# InternVL3.5-1B — QHexRT NPU bundle (Hexagon v79)
+
+Precompiled **InternVL3.5-1B** vision-language model for the **QHexRT** runtime on Qualcomm
+Hexagon **v79** (Snapdragon 8 Elite / SM8750, e.g. Galaxy S25). Image → caption runs **entirely on
+the NPU**: the InternViT-300M vision encoder and the full Qwen3-0.6B language model are
+`qnn_context_binary` graphs; the host does only light glue (image resize, patch-embed, pixel-shuffle
+projector, tokenize/splice, sampling). Device-validated end-to-end on diverse photos.
+
+The model is driven by QHexRT's declarative **manifest plan-interpreter** (`Session::generate` via the
+`internvl_generate` host-op) — the same engine that runs the other `*_HNPU` bundles.
+
+## Contents (`v79/`)
+| file | what | size |
+|------|------|------|
+| `internvl3_5-1b.json` | QHexRT manifest (vlm family, `internvl_generate` plan) | — |
+| `ivl_enc2.bin` | InternViT-300M vision encoder — 24-layer, **fp16**, 2D `[seq,dim]` I/O | 621 MB |
+| `ivl_pf.bin` | Qwen3-0.6B **prefill** graph, fixed `S=310` (**fp16**) | 898 MB |
+| `ivl_dec_w4.bin` | Qwen3-0.6B **decode** step (1 token), **W4A16** | 225 MB |
+| `ivl_lmh_q.bin` | untied **lm-head**, **W8A16** (int8) → logits over 151936 vocab | 158 MB |
+| `ivl_embed_f16.bin` | token embedding table `[151936,1024]` f16 (mmap, host lookup) | 311 MB |
+| `tokenizer.json` | Qwen3 tokenizer (vocab 151936) | 11 MB |
+| `vlm/` | host vision preproc: Conv2d patch-embed, pos-embed, layer-norm + pixel-shuffle MLP projector, prompt template | 27 MB |
+
+Total ≈ **2.25 GB** on disk; ≈ **2.0 GB** peak device RSS at runtime (weights are ion-shared host↔DSP,
+not duplicated).
+
+## Run (QHexRT CLI)
+```bash
+hf download runanywhere/internvl3_5_1b_HNPU --local-dir internvl3_5_1b_HNPU
+# qhx_generate comes from a QHexRT build; QNN libs from the QAIRT SDK (lib/aarch64-android) + v79 HTP skel.
+adb push internvl3_5_1b_HNPU/v79 /data/local/tmp/wq/internvl
+adb push my_photo.jpg /data/local/tmp/wq/internvl/
+
+adb shell "cd /data/local/tmp/wq && LD_LIBRARY_PATH=. ADSP_LIBRARY_PATH=. \
+  ./qhx_generate internvl/internvl3_5-1b.json libQnnHtp.so libQnnSystem.so internvl \
+  60 'Describe this image in detail.' internvl/my_photo.jpg"
+# -> text: "The image shows ..."
+```
+- **Image**: any aspect ratio / format (JPEG/PNG/…); center-cropped + resized to 448×448 host-side
+  (256 image tokens / 1 tile).
+- **Prompt is fixed**: the shipped prefill graph is pinned to `S=310`, which is the tokenized
+  `"Describe this image in detail."` (+ 256 image tokens + chat template). Pass exactly that string
+  (or `''` to use it as the default). Other prompts need a re-exported prefill at the new sequence length.
+
+## Performance (measured, v79 / S25)
+| stage | time |
+|-------|------|
+| vision encode (one-time) | **181 ms** |
+| prefill / TTFT | **101 ms** |
+| decode | **~28 ms/token** on the NPU graphs |
+
+End-to-end wall time includes single-threaded host preproc (patch-embed + projector, ~1.5 s one-time);
+the NPU compute itself is the table above.
+
+## Notes
+- Arch: **v79 only** — context binaries are dsp-arch-pinned. Other arches must be re-exported.
+- No custom op-package needed — all graphs are pure-native HTP (HVX/HMX).
+- **Precision**: vision fp16, prefill fp16, decode **W4A16**, lm-head **W8A16**. (Prefill/vision can be
+  further quantized to shrink the footprint; this bundle ships the validated balance.)
+- **HTP gotcha baked in**: all graph I/O is 2D `[seq,dim]` — HTP mis-lays-out 3D `[1,seq,dim]` tensors.
+  The Conv2d patch-embed and pixel-shuffle are host-side for the same reason.
+- Source model: [`OpenGVLab/InternVL3_5-1B`](https://huggingface.co/OpenGVLab/InternVL3_5-1B)
+  (InternViT-300M + Qwen3-0.6B), exported and compiled with QAIRT 2.45 for
+  `qualcomm-snapdragon-8-elite-for-galaxy`.