v75 (S24/SM8650, soc 57): DeepSeek-R1-Distill-Qwen-1.5B fp16 bundle — export 8/8 (runtime needs >8GB RAM)
#2
by flying101 - opened
fp16 decode+prefill+lmhead+embed (~5.8G, same layout as v79/v81), QAIRT 2.47 soc 57. Export gate 8/8. Runtime needs >8GB RAM (fp16 dual 2.5G graphs).
Adds the Hexagon v75 bundle under (same ~5.8G fp16 layout as v79/v81).
- Files: DeepSeek-R1-Distill-Qwen-1.5B.json, decode_f16.bin (2.5G), prefill_f16.bin (2.5G), lmhead_f16.bin, embed_f16.bin, tokenizer.json
- Precision: fp16 throughout (KB: W8 degrades DeepSeek-1.5B quality)
- Compile: QAIRT 2.47.0, v75/soc 57; export gate decode 8/8 match HF greedy
- Device note: all 3 graphs load on v75, but the dual 2.5G fp16 decode+prefill exceed the 8GB Galaxy S24 RAM at runtime (adb drop). Runs on higher-RAM v75 devices; on 8GB devices a W8 build is needed. Published for parity with v79/v81.
flying101 changed pull request title from Add Hexagon v75 (Galaxy S24 / SM8650, soc 57) bundle to v75 (S24/SM8650, soc 57): DeepSeek-R1-Distill-Qwen-1.5B fp16 bundle — export 8/8 (runtime needs >8GB RAM)
v75 bundle details: fp16 decode (2.5G) + prefill (2.5G) + lm-head + embed (~5.8G, same layout as v79/v81; KB: W8 degrades this model). QAIRT 2.47, v75/soc 57, export decode 8/8. Device: 3 graphs load on v75 but dual 2.5G fp16 graphs exceed the 8GB S24 RAM (adb drop); runs on higher-RAM devices.
flying101 changed pull request status to open
sanmonga22 changed pull request status to merged