v75 (Galaxy S24 / SM8650, soc_model 57): Qwen3.5-2B W8-decode bundle — Plane A 6/6, 10.5 tok/s
#2
by flying101 - opened
W8 decode + fp16 lmhead/embed, compiled QAIRT 2.47 soc_model 57. Device-validated on S24 (4/4 canonical prompts, 10.5 tok/s).
flying101 changed pull request status to open
Adds the Hexagon v75 flat bundle under (Snapdragon 8 Gen 3 / Galaxy S24 class).
- Files: qwen3.5-2b-1024.json, qwen352b_decode_w8.bin (W8, 1.3G), qwen352b_lmhead_f16.bin, qwen352b_embed_f16.bin, tokenizer.json
- Precision: W8A16 weight-only decode + fp16 lm-head/embed; GatedDeltaNet hybrid, host-op qwen3_5_generate
- Compile: QAIRT 2.47.0, dsp_arch v75, soc_model 57; export gate decode 8/8 match HF greedy
- Device (Plane A, raw qhx_generate on S24 RFCX200WNWW): 6/6 standardized prompts (Paris/Tokyo/four/cold/Jupiter/H2O), ~10.5 tok/s decode, thinking-mode output parsed correctly
flying101 changed pull request title from Add Hexagon v75 (Galaxy S24 / SM8650) bundle to v75 (Galaxy S24 / SM8650, soc_model 57): Qwen3.5-2B W8-decode bundle — Plane A 6/6, 10.5 tok/s
v75 bundle details: W8 decode (1.3G) + fp16 lm-head/embed; GatedDeltaNet hybrid, host-op qwen3_5_generate. QAIRT 2.47, v75/soc 57, export decode 8/8. Plane A on S24: 6/6 prompts (Paris/Tokyo/four/cold/Jupiter/H2O), ~10.5 tok/s.
sanmonga22 changed pull request status to merged