File size: 5,311 Bytes
39ae3a9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | ---
license: other
license_name: stabilityai-ai-community
license_link: https://huggingface.co/stabilityai/sdxl-turbo/blob/main/LICENSE.md
tags:
- stable-diffusion-xl
- sdxl-turbo
- qnn
- hexagon
- npu
- w8a16
- text-to-image
- tokforge
pipeline_tag: text-to-image
library_name: qnn
inference: false
---
# TokForge SDXL-Turbo · Hexagon NPU (W8A16)
**PRIVATE** model bundle for the **TokForge** Android app (`dev.tokforge`). This repo hosts the
**SDXL-Turbo** image-generation pipeline compiled to **QNN context binaries** for the
**Qualcomm Hexagon DSP (HTP)**, so TokForge can fetch the SDXL-NPU "Faithful" backend from scratch
(previously hand-staged on devices only).
Sibling of [`darkmaniac7/TokForge-SD15-QNN-NPU`](https://huggingface.co/darkmaniac7/TokForge-SD15-QNN-NPU)
(SD1.5). Same non-root `untrusted_app` cDSP recipe; this is the higher-quality 1024px tier.
## What this is
- **Model:** SDXL-Turbo (few-step distilled SDXL), native **1024×1024** (128×128 latent).
- **Quantization:** **W8A16** (8-bit weights, 16-bit activations) on the Hexagon HTP.
- **Runtime:** **QAIRT 2.40**. Bin set is **V75-native** (`libQnnHtpV75Skel.so`).
- **Variant:** **fp16 text encoders + TAESDXL** tiny-VAE (see below).
- **Footprint:** **~4.3 GB** on disk (the on-device fresh download is this big).
## Pipeline / variant
This bundle is the **fp16-encoders + TAESDXL** variant the `libsdxl_qnn_driver` runs end-to-end
(no host orchestration beyond the emb-MLP):
| Stage | File | Where it runs |
|-------|------|---------------|
| CLIP-L text encoder | `text_encoder_1_fp16.bin` | DSP (fp16) → `last_hidden [1,77,768]` |
| OpenCLIP-bigG text encoder (+proj) | `text_encoder_2_fp16.bin` | DSP (fp16) → `last_hidden [1,77,1280]` + `pooled [1,1280]` |
| text_embedding | concat → `[1,77,2048]` | driver |
| combined-emb MLP | `sdxl_emb_mlp.bin` | **host CPU** → `emb [1,1280]` per step |
| UNet (EulerAncestral, 4 steps, guidance 0) | `unet.bin` (2.6 GB) | DSP |
| VAE decode (1024px) | `taesdxl_decoder.mnn` (2.4 MB) | **CPU** (MNN tiny-VAE) |
**Why the host emb-MLP:** qairt-converter 2.40 constant-folds SDXL's two sinusoidal paths
(`time_proj` and `add_time_proj`). The driver dodges both by precomputing the **combined**
`emb = time_embedding(time_proj(t)) + add_embedding(cat(pooled, add_time_proj(time_ids)))` on host
(weights in `sdxl_emb_mlp.bin`) and feeding it as ONE float UNet input, so no foldable path remains.
**Why TAESDXL CPU VAE:** it is the **only validated** VAE path. The DSP VAE is blocked at 1024px on
V75 (unsigned-PD HTP cap) and the OpenCL/Vulkan VAE SIGSEGVs Adreno in `Session::resize`. The driver
always selects `--vae_mnn` + `SDXL_VAE_BACKEND=cpu`.
**SDXL-Turbo is guidance-free** (`guidance_scale 0`): NO CFG → one UNet pass per step. EulerAncestral,
4 steps native (6 for quality), VAE scaling factor **0.13025**.
## Arch coverage & verification
| Set | dsp_arch | Skel | Verified |
|-----|----------|------|----------|
| `ours/v75` | 75 (V75) | `libQnnHtpV75Skel.so` | **YES** — device-verified coherent+crisp on Lenovo **SM8650/V75** (.69) |
**Forward-compat:** a lower-arch bin runs on a higher-arch DSP. The **V75-native** set is the
shipping set for **V73 → V81**, and was **verified by forward-compat on V81** (RedMagic **SM8850**
handsets .82 and .80, coherent+crisp). **Native V81/V79 SDXL bins are NOT separately built** — V75-native
is the single own-built set today.
| SoC class | Hexagon | Status |
|-----------|---------|--------|
| 8 Gen 2 / 8s Gen 3 (SM8550/SM8635) | V73 | covered by V75-native forward-compat (untested on device) |
| 8 Gen 3 (SM8650) | V75 | **device-verified** |
| 8 Elite (SM8750) | V79 | forward-compat (untested on V79 silicon) |
| 8 Elite Gen 5 (SM8850) | V81 | **forward-compat verified** (.82, .80) |
## Provenance
Clean-room conversion, our own pipeline (no third-party NPU bins):
1. **Base:** `stabilityai/sdxl-turbo` (fp16 safetensors) → ONNX export (UNet emb-input/fold-proofed
graph, dual CLIP encoders, TAESDXL decoder).
2. **Quantize:** **W8A16** via QAIRT (calibration set), graph name `model`.
3. **Compile:** `qnn-context-binary-generator` → V75 HTP context binaries (**QAIRT 2.40**).
4. **VAE:** TAESDXL exported to a CPU MNN model (`taesdxl_decoder.mnn`).
## License
**SDXL-Turbo — Stability AI Community License** (`stabilityai/sdxl-turbo`).
- Commercial use **permitted** for organizations/individuals with **< $1M USD annual revenue**.
- Above $1M revenue requires a **Stability AI Enterprise license**.
- **Attribution + license text must be retained** with the model.
- This is **NOT** OpenRAIL-M (that covers the sibling SD1.5 bundle).
Text encoder bases: OpenCLIP bigG (MIT-class), OpenAI CLIP-L (MIT).
## Files
See `manifest.json` for the authoritative file set with per-file md5 + size. The app reads
`manifest.json`, resolves the device Hexagon arch, downloads the matching set into
`filesDir/image_models/SDXL-QNN-NPU/`, and the `libsdxl_qnn_driver` loads the bins flat from there.
**Driver-required files** (every one must be present + non-empty for a usable bundle):
`unet.bin`, `text_encoder_1_fp16.bin`, `text_encoder_2_fp16.bin`, `sdxl_emb_mlp.bin`,
`taesdxl_decoder.mnn`, `tokenizer/{vocab.json,merges.txt}`, `tokenizer_2/{vocab.json,merges.txt}`.
|