Add SDXL-Turbo W8A16 Hexagon NPU bundle (V75-native, fwd-compat V79/V81) + manifest

39ae3a9 verified 2 months ago

5.31 kB

license: other
license_name: stabilityai-ai-community
license_link: https://huggingface.co/stabilityai/sdxl-turbo/blob/main/LICENSE.md
tags:
  - stable-diffusion-xl
  - sdxl-turbo
  - qnn
  - hexagon
  - npu
  - w8a16
  - text-to-image
  - tokforge
pipeline_tag: text-to-image
library_name: qnn
inference: false

TokForge SDXL-Turbo · Hexagon NPU (W8A16)

PRIVATE model bundle for the TokForge Android app (dev.tokforge). This repo hosts the SDXL-Turbo image-generation pipeline compiled to QNN context binaries for the Qualcomm Hexagon DSP (HTP), so TokForge can fetch the SDXL-NPU "Faithful" backend from scratch (previously hand-staged on devices only).

Sibling of darkmaniac7/TokForge-SD15-QNN-NPU (SD1.5). Same non-root untrusted_app cDSP recipe; this is the higher-quality 1024px tier.

What this is

Model: SDXL-Turbo (few-step distilled SDXL), native 1024×1024 (128×128 latent).
Quantization: W8A16 (8-bit weights, 16-bit activations) on the Hexagon HTP.
Runtime: QAIRT 2.40. Bin set is V75-native (libQnnHtpV75Skel.so).
Variant: fp16 text encoders + TAESDXL tiny-VAE (see below).
Footprint: ~4.3 GB on disk (the on-device fresh download is this big).

Pipeline / variant

This bundle is the fp16-encoders + TAESDXL variant the libsdxl_qnn_driver runs end-to-end (no host orchestration beyond the emb-MLP):

Stage	File	Where it runs
CLIP-L text encoder	`text_encoder_1_fp16.bin`	DSP (fp16) → `last_hidden [1,77,768]`
OpenCLIP-bigG text encoder (+proj)	`text_encoder_2_fp16.bin`	DSP (fp16) → `last_hidden [1,77,1280]` + `pooled [1,1280]`
text_embedding	concat → `[1,77,2048]`	driver
combined-emb MLP	`sdxl_emb_mlp.bin`	host CPU → `emb [1,1280]` per step
UNet (EulerAncestral, 4 steps, guidance 0)	`unet.bin` (2.6 GB)	DSP
VAE decode (1024px)	`taesdxl_decoder.mnn` (2.4 MB)	CPU (MNN tiny-VAE)

Why the host emb-MLP: qairt-converter 2.40 constant-folds SDXL's two sinusoidal paths (time_proj and add_time_proj). The driver dodges both by precomputing the combined emb = time_embedding(time_proj(t)) + add_embedding(cat(pooled, add_time_proj(time_ids))) on host (weights in sdxl_emb_mlp.bin) and feeding it as ONE float UNet input, so no foldable path remains.

Why TAESDXL CPU VAE: it is the only validated VAE path. The DSP VAE is blocked at 1024px on V75 (unsigned-PD HTP cap) and the OpenCL/Vulkan VAE SIGSEGVs Adreno in Session::resize. The driver always selects --vae_mnn + SDXL_VAE_BACKEND=cpu.

SDXL-Turbo is guidance-free (guidance_scale 0): NO CFG → one UNet pass per step. EulerAncestral, 4 steps native (6 for quality), VAE scaling factor 0.13025.

Arch coverage & verification

Set	dsp_arch	Skel	Verified
`ours/v75`	75 (V75)	`libQnnHtpV75Skel.so`	YES — device-verified coherent+crisp on Lenovo SM8650/V75 (.69)

Forward-compat: a lower-arch bin runs on a higher-arch DSP. The V75-native set is the shipping set for V73 → V81, and was verified by forward-compat on V81 (RedMagic SM8850 handsets .82 and .80, coherent+crisp). Native V81/V79 SDXL bins are NOT separately built — V75-native is the single own-built set today.

SoC class	Hexagon	Status
8 Gen 2 / 8s Gen 3 (SM8550/SM8635)	V73	covered by V75-native forward-compat (untested on device)
8 Gen 3 (SM8650)	V75	device-verified
8 Elite (SM8750)	V79	forward-compat (untested on V79 silicon)
8 Elite Gen 5 (SM8850)	V81	forward-compat verified (.82, .80)

Provenance

Clean-room conversion, our own pipeline (no third-party NPU bins):

Base: stabilityai/sdxl-turbo (fp16 safetensors) → ONNX export (UNet emb-input/fold-proofed graph, dual CLIP encoders, TAESDXL decoder).
Quantize: W8A16 via QAIRT (calibration set), graph name model.
Compile: qnn-context-binary-generator → V75 HTP context binaries (QAIRT 2.40).
VAE: TAESDXL exported to a CPU MNN model (taesdxl_decoder.mnn).

License

SDXL-Turbo — Stability AI Community License (stabilityai/sdxl-turbo).

Commercial use permitted for organizations/individuals with < $1M USD annual revenue.
Above $1M revenue requires a Stability AI Enterprise license.
Attribution + license text must be retained with the model.
This is NOT OpenRAIL-M (that covers the sibling SD1.5 bundle).

Text encoder bases: OpenCLIP bigG (MIT-class), OpenAI CLIP-L (MIT).

Files

See manifest.json for the authoritative file set with per-file md5 + size. The app reads manifest.json, resolves the device Hexagon arch, downloads the matching set into filesDir/image_models/SDXL-QNN-NPU/, and the libsdxl_qnn_driver loads the bins flat from there.

Driver-required files (every one must be present + non-empty for a usable bundle): unet.bin, text_encoder_1_fp16.bin, text_encoder_2_fp16.bin, sdxl_emb_mlp.bin, taesdxl_decoder.mnn, tokenizer/{vocab.json,merges.txt}, tokenizer_2/{vocab.json,merges.txt}.