Aman0runanywhere commited on
Commit
75bd23f
·
verified ·
1 Parent(s): bd54ad6

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +7 -8
README.md CHANGED
@@ -28,9 +28,9 @@ The model is driven by QHexRT's declarative **manifest plan-interpreter** (`Sess
28
  ## Contents (`v79/`)
29
  | file | what | size |
30
  |------|------|------|
31
- | `internvl3_5-1b.json` | QHexRT manifest (vlm family, `internvl_generate` plan) | — |
32
  | `ivl_enc2.bin` | InternViT-300M vision encoder — 24-layer, **fp16**, 2D `[seq,dim]` I/O | 621 MB |
33
- | `ivl_pf.bin` | Qwen3-0.6B **prefill** graph, fixed `S=310` (**fp16**) | 898 MB |
34
  | `ivl_dec_w4.bin` | Qwen3-0.6B **decode** step (1 token), **W4A16** | 225 MB |
35
  | `ivl_lmh_q.bin` | untied **lm-head**, **W8A16** (int8) → logits over 151936 vocab | 158 MB |
36
  | `ivl_embed_f16.bin` | token embedding table `[151936,1024]` f16 (mmap, host lookup) | 311 MB |
@@ -48,15 +48,14 @@ adb push internvl3_5_1b_HNPU/v79 /data/local/tmp/wq/internvl
48
  adb push my_photo.jpg /data/local/tmp/wq/internvl/
49
 
50
  adb shell "cd /data/local/tmp/wq && LD_LIBRARY_PATH=. ADSP_LIBRARY_PATH=. \
51
- ./qhx_generate internvl/internvl3_5-1b.json libQnnHtp.so libQnnSystem.so internvl \
52
- 60 'Describe this image in detail.' internvl/my_photo.jpg"
53
- # -> text: "The image shows ..."
54
  ```
55
  - **Image**: any aspect ratio / format (JPEG/PNG/…); center-cropped + resized to 448×448 host-side
56
  (256 image tokens / 1 tile).
57
- - **Prompt is fixed**: the shipped prefill graph is pinned to `S=310`, which is the tokenized
58
- `"Describe this image in detail."` (+ 256 image tokens + chat template). Pass exactly that string
59
- (or `''` to use it as the default). Other prompts need a re-exported prefill at the new sequence length.
60
 
61
  ## Performance (measured, v79 / S25)
62
  | stage | time |
 
28
  ## Contents (`v79/`)
29
  | file | what | size |
30
  |------|------|------|
31
+ | `internvl3_5-1b-512.json` | QHexRT manifest (vlm family, `internvl_generate` plan) | — |
32
  | `ivl_enc2.bin` | InternViT-300M vision encoder — 24-layer, **fp16**, 2D `[seq,dim]` I/O | 621 MB |
33
+ | `ivl_pf_512.bin` | Qwen3-0.6B **batched prefill** graph, AR=512 (padded) — **arbitrary prompts** up to ~256 text tokens (**fp16**) | 890 MB |
34
  | `ivl_dec_w4.bin` | Qwen3-0.6B **decode** step (1 token), **W4A16** | 225 MB |
35
  | `ivl_lmh_q.bin` | untied **lm-head**, **W8A16** (int8) → logits over 151936 vocab | 158 MB |
36
  | `ivl_embed_f16.bin` | token embedding table `[151936,1024]` f16 (mmap, host lookup) | 311 MB |
 
48
  adb push my_photo.jpg /data/local/tmp/wq/internvl/
49
 
50
  adb shell "cd /data/local/tmp/wq && LD_LIBRARY_PATH=. ADSP_LIBRARY_PATH=. \
51
+ ./qhx_generate internvl/internvl3_5-1b-512.json libQnnHtp.so libQnnSystem.so internvl \
52
+ 60 'What animals are in this picture and what are they doing?' internvl/my_photo.jpg"
53
+ # -> text: "In the image, there are two cats lying on a pink couch ..."
54
  ```
55
  - **Image**: any aspect ratio / format (JPEG/PNG/…); center-cropped + resized to 448×448 host-side
56
  (256 image tokens / 1 tile).
57
+ - **Arbitrary prompts**: the padded `AR=512` prefill accepts any prompt up to ~256 text tokens (image is
58
+ fixed at 256 tokens). Ask real questions, not just a fixed caption string.
 
59
 
60
  ## Performance (measured, v79 / S25)
61
  | stage | time |