README.md

metadata

license: apache-2.0
tags:
  - qhexrt
  - internvl
  - vlm
  - image-text-to-text
  - hexagon
  - npu
  - qnn
base_model: OpenGVLab/InternVL3_5-1B
pipeline_tag: image-text-to-text
language:
  - en

InternVL3.5-1B — QHexRT NPU bundle (Hexagon v79)

Precompiled InternVL3.5-1B vision-language model for the QHexRT runtime on Qualcomm Hexagon v79 (Snapdragon 8 Elite / SM8750, e.g. Galaxy S25). Image → caption runs entirely on the NPU: the InternViT-300M vision encoder and the full Qwen3-0.6B language model are qnn_context_binary graphs; the host does only light glue (image resize, patch-embed, pixel-shuffle projector, tokenize/splice, sampling). Device-validated end-to-end on diverse photos.

The model is driven by QHexRT's declarative manifest plan-interpreter (Session::generate via the internvl_generate host-op) — the same engine that runs the other *_HNPU bundles.

Contents (v79/)

file	what	size
internvl3_5-1b-512.json	QHexRT manifest (vlm family, internvl_generate plan)	—
ivl_enc2.bin	InternViT-300M vision encoder — 24-layer, fp16, 2D [seq,dim] I/O	621 MB
ivl_pf_512.bin	Qwen3-0.6B batched prefill graph, AR=512 (padded) — arbitrary prompts up to ~256 text tokens (fp16)	890 MB
ivl_dec_w4.bin	Qwen3-0.6B decode step (1 token), W4A16	225 MB
ivl_lmh_q.bin	untied lm-head, W8A16 (int8) → logits over 151936 vocab	158 MB
ivl_embed_f16.bin	token embedding table [151936,1024] f16 (mmap, host lookup)	311 MB
tokenizer.json	Qwen3 tokenizer (vocab 151936)	11 MB
vlm/	host vision preproc: Conv2d patch-embed, pos-embed, layer-norm + pixel-shuffle MLP projector, prompt template	27 MB

Total ≈ 2.25 GB on disk; ≈ 2.0 GB peak device RSS at runtime (weights are ion-shared host↔DSP, not duplicated).

Run (QHexRT CLI)

hf download runanywhere/internvl3_5_1b_HNPU --local-dir internvl3_5_1b_HNPU
# qhx_generate comes from a QHexRT build; QNN libs from the QAIRT SDK (lib/aarch64-android) + v79 HTP skel.
adb push internvl3_5_1b_HNPU/v79 /data/local/tmp/wq/internvl
adb push my_photo.jpg /data/local/tmp/wq/internvl/

adb shell "cd /data/local/tmp/wq && LD_LIBRARY_PATH=. ADSP_LIBRARY_PATH=. \
  ./qhx_generate internvl/internvl3_5-1b-512.json libQnnHtp.so libQnnSystem.so internvl \
  60 'What animals are in this picture and what are they doing?' internvl/my_photo.jpg"
# -> text: "In the image, there are two cats lying on a pink couch ..."

• Image: any aspect ratio / format (JPEG/PNG/…); center-cropped + resized to 448×448 host-side (256 image tokens / 1 tile).
• Arbitrary prompts: the padded AR=512 prefill accepts any prompt up to ~256 text tokens (image is fixed at 256 tokens). Ask real questions, not just a fixed caption string.

Performance (measured, v79 / S25)

stage	time
vision encode (one-time)	181 ms
prefill / TTFT	101 ms
decode	~28 ms/token on the NPU graphs

End-to-end wall time includes single-threaded host preproc (patch-embed + projector, ~1.5 s one-time); the NPU compute itself is the table above.

Notes

• Arch: v79 only — context binaries are dsp-arch-pinned. Other arches must be re-exported.
• No custom op-package needed — all graphs are pure-native HTP (HVX/HMX).
• Precision: vision fp16, prefill fp16, decode W4A16, lm-head W8A16. (Prefill/vision can be further quantized to shrink the footprint; this bundle ships the validated balance.)
• HTP gotcha baked in: all graph I/O is 2D [seq,dim] — HTP mis-lays-out 3D [1,seq,dim] tensors. The Conv2d patch-embed and pixel-shuffle are host-side for the same reason.
• Source model: OpenGVLab/InternVL3_5-1B (InternViT-300M + Qwen3-0.6B), exported and compiled with QAIRT 2.45 for qualcomm-snapdragon-8-elite-for-galaxy.

v81 (SM8850 / soc_model 87)

Device-validated on SM8850: image->caption tracks the image — e.g. an orange dahlia -> "The image features a close-up of a single, large, orange flower with a textured, layered appearance. The flower has a circular shape with numerous petals radiating from a central point...". Vision 147 ms, prefill TTFT 115 ms, decode 18.4 ms/tok. The v81 recipe was completed by reconstructing the two graphs the internvl_generate host-op needs but the recipe lacked: a batched-prefill graph (ivl_pf, AR=512, GQA) and a lm-head graph (ivl_lmh) — both gated vs the HF gold first token. All graphs f16 (f16 I/O), DLC route + O3.

Files (v81/)

internvl3_5-1b.json · ivl_enc_f16.bin (InternViT-300M vision) · ivl_pf_f16.bin (batched prefill) · ivl_dec_f16.bin (AR=1 decode) · ivl_lmh_f16.bin (lm-head) · ivl_embed_f16.bin · tokenizer.json · vlm/ (host patch/pos-embed + pixel-shuffle MLP projector weights + prompt template).