Add SDXL-Turbo W8A16 Hexagon NPU bundle (V75-native, fwd-compat V79/V81) + manifest

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+v75/taesdxl_decoder.mnn filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,110 @@
+---
+license: other
+license_name: stabilityai-ai-community
+license_link: https://huggingface.co/stabilityai/sdxl-turbo/blob/main/LICENSE.md
+tags:
+  - stable-diffusion-xl
+  - sdxl-turbo
+  - qnn
+  - hexagon
+  - npu
+  - w8a16
+  - text-to-image
+  - tokforge
+pipeline_tag: text-to-image
+library_name: qnn
+inference: false
+---
+
+# TokForge SDXL-Turbo · Hexagon NPU (W8A16)
+
+**PRIVATE** model bundle for the **TokForge** Android app (`dev.tokforge`). This repo hosts the
+**SDXL-Turbo** image-generation pipeline compiled to **QNN context binaries** for the
+**Qualcomm Hexagon DSP (HTP)**, so TokForge can fetch the SDXL-NPU "Faithful" backend from scratch
+(previously hand-staged on devices only).
+
+Sibling of [`darkmaniac7/TokForge-SD15-QNN-NPU`](https://huggingface.co/darkmaniac7/TokForge-SD15-QNN-NPU)
+(SD1.5). Same non-root `untrusted_app` cDSP recipe; this is the higher-quality 1024px tier.
+
+## What this is
+
+- **Model:** SDXL-Turbo (few-step distilled SDXL), native **1024×1024** (128×128 latent).
+- **Quantization:** **W8A16** (8-bit weights, 16-bit activations) on the Hexagon HTP.
+- **Runtime:** **QAIRT 2.40**. Bin set is **V75-native** (`libQnnHtpV75Skel.so`).
+- **Variant:** **fp16 text encoders + TAESDXL** tiny-VAE (see below).
+- **Footprint:** **~4.3 GB** on disk (the on-device fresh download is this big).
+
+## Pipeline / variant
+
+This bundle is the **fp16-encoders + TAESDXL** variant the `libsdxl_qnn_driver` runs end-to-end
+(no host orchestration beyond the emb-MLP):
+
+| Stage | File | Where it runs |
+|-------|------|---------------|
+| CLIP-L text encoder | `text_encoder_1_fp16.bin` | DSP (fp16) → `last_hidden [1,77,768]` |
+| OpenCLIP-bigG text encoder (+proj) | `text_encoder_2_fp16.bin` | DSP (fp16) → `last_hidden [1,77,1280]` + `pooled [1,1280]` |
+| text_embedding | concat → `[1,77,2048]` | driver |
+| combined-emb MLP | `sdxl_emb_mlp.bin` | **host CPU** → `emb [1,1280]` per step |
+| UNet (EulerAncestral, 4 steps, guidance 0) | `unet.bin` (2.6 GB) | DSP |
+| VAE decode (1024px) | `taesdxl_decoder.mnn` (2.4 MB) | **CPU** (MNN tiny-VAE) |
+
+**Why the host emb-MLP:** qairt-converter 2.40 constant-folds SDXL's two sinusoidal paths
+(`time_proj` and `add_time_proj`). The driver dodges both by precomputing the **combined**
+`emb = time_embedding(time_proj(t)) + add_embedding(cat(pooled, add_time_proj(time_ids)))` on host
+(weights in `sdxl_emb_mlp.bin`) and feeding it as ONE float UNet input, so no foldable path remains.
+
+**Why TAESDXL CPU VAE:** it is the **only validated** VAE path. The DSP VAE is blocked at 1024px on
+V75 (unsigned-PD HTP cap) and the OpenCL/Vulkan VAE SIGSEGVs Adreno in `Session::resize`. The driver
+always selects `--vae_mnn` + `SDXL_VAE_BACKEND=cpu`.
+
+**SDXL-Turbo is guidance-free** (`guidance_scale 0`): NO CFG → one UNet pass per step. EulerAncestral,
+4 steps native (6 for quality), VAE scaling factor **0.13025**.
+
+## Arch coverage & verification
+
+| Set | dsp_arch | Skel | Verified |
+|-----|----------|------|----------|
+| `ours/v75` | 75 (V75) | `libQnnHtpV75Skel.so` | **YES** — device-verified coherent+crisp on Lenovo **SM8650/V75** (.69) |
+
+**Forward-compat:** a lower-arch bin runs on a higher-arch DSP. The **V75-native** set is the
+shipping set for **V73 → V81**, and was **verified by forward-compat on V81** (RedMagic **SM8850**
+handsets .82 and .80, coherent+crisp). **Native V81/V79 SDXL bins are NOT separately built** — V75-native
+is the single own-built set today.
+
+| SoC class | Hexagon | Status |
+|-----------|---------|--------|
+| 8 Gen 2 / 8s Gen 3 (SM8550/SM8635) | V73 | covered by V75-native forward-compat (untested on device) |
+| 8 Gen 3 (SM8650) | V75 | **device-verified** |
+| 8 Elite (SM8750) | V79 | forward-compat (untested on V79 silicon) |
+| 8 Elite Gen 5 (SM8850) | V81 | **forward-compat verified** (.82, .80) |
+
+## Provenance
+
+Clean-room conversion, our own pipeline (no third-party NPU bins):
+
+1. **Base:** `stabilityai/sdxl-turbo` (fp16 safetensors) → ONNX export (UNet emb-input/fold-proofed
+   graph, dual CLIP encoders, TAESDXL decoder).
+2. **Quantize:** **W8A16** via QAIRT (calibration set), graph name `model`.
+3. **Compile:** `qnn-context-binary-generator` → V75 HTP context binaries (**QAIRT 2.40**).
+4. **VAE:** TAESDXL exported to a CPU MNN model (`taesdxl_decoder.mnn`).
+
+## License
+
+**SDXL-Turbo — Stability AI Community License** (`stabilityai/sdxl-turbo`).
+
+- Commercial use **permitted** for organizations/individuals with **< $1M USD annual revenue**.
+- Above $1M revenue requires a **Stability AI Enterprise license**.
+- **Attribution + license text must be retained** with the model.
+- This is **NOT** OpenRAIL-M (that covers the sibling SD1.5 bundle).
+
+Text encoder bases: OpenCLIP bigG (MIT-class), OpenAI CLIP-L (MIT).
+
+## Files
+
+See `manifest.json` for the authoritative file set with per-file md5 + size. The app reads
+`manifest.json`, resolves the device Hexagon arch, downloads the matching set into
+`filesDir/image_models/SDXL-QNN-NPU/`, and the `libsdxl_qnn_driver` loads the bins flat from there.
+
+**Driver-required files** (every one must be present + non-empty for a usable bundle):
+`unet.bin`, `text_encoder_1_fp16.bin`, `text_encoder_2_fp16.bin`, `sdxl_emb_mlp.bin`,
+`taesdxl_decoder.mnn`, `tokenizer/{vocab.json,merges.txt}`, `tokenizer_2/{vocab.json,merges.txt}`.

manifest.json

@@ -0,0 +1,171 @@
+{
+  "schema": "tokforge-sdxl-qnn-npu/manifest@1",
+  "model": "sdxl-turbo",
+  "variant": "fp16-encoders + TAESDXL",
+  "license": "Stability AI Community License (SDXL-Turbo)",
+  "license_note": "Commercial use permitted for orgs/individuals with < $1M USD annual revenue; above that requires a Stability AI Enterprise license. Attribution + license text must be retained with the model. NOT OpenRAIL (that was SD1.5).",
+  "generated": "2026-06-05T03:08:58Z",
+  "repo": "darkmaniac7/TokForge-SDXL-QNN-NPU (PRIVATE)",
+  "notes": [
+    "App reads the device Hexagon arch (dsp_arch) and picks the matching set. SDXL ships ONE own-built set today: V75-native (ours).",
+    "Forward-compat: a lower-arch bin runs on a higher-arch DSP. The V75-native SDXL set is forward-compat to V79/V81 silicon (verified on the V81 SM8850 handset, .82 + .80). NATIVE V81/V79 SDXL bins are NOT separately built yet.",
+    "source=ours: our own clean-room SDXL-Turbo -> W8A16 -> QAIRT 2.40 conversion, drop-in for our libsdxl_qnn_driver (graph name 'model', host combined-emb input). There is no AI-Hub SDXL set in this repo.",
+    "verified=true means run on real silicon and confirmed coherent+crisp. verified=false = produced + offline-validated but NEEDS a handset before NPU-enabling.",
+    "VAE path: TAESDXL CPU tiny-VAE (taesdxl_decoder.mnn, 2.4 MB) via --vae_mnn / SDXL_VAE_BACKEND=cpu. The DSP VAE is blocked at 1024px on V75 (unsigned-PD HTP cap) and the OpenCL/Vulkan VAE SIGSEGVs Adreno in Session::resize. CPU TAESDXL is the ONLY validated VAE path.",
+    "On-device fresh download is ~4.3 GB (4,308,394,979 bytes): unet.bin 2.6 GB + text_encoder_2_fp16 1.4 GB + text_encoder_1_fp16 235 MB + sdxl_emb_mlp 29 MB + taesdxl 2.4 MB + tokenizers/config."
+  ],
+  "arch_to_socs": {
+    "73": {
+      "name": "Hexagon V73",
+      "socs": [
+        "SM8550 (Snapdragon 8 Gen 2)",
+        "SM8635 (8s Gen 3)",
+        "SM7475 (7+ Gen 3)"
+      ]
+    },
+    "75": {
+      "name": "Hexagon V75",
+      "socs": [
+        "SM8650 (Snapdragon 8 Gen 3)",
+        "SM7675 (7 Gen 4)"
+      ]
+    },
+    "79": {
+      "name": "Hexagon V79",
+      "socs": [
+        "SM8750 (Snapdragon 8 Elite)",
+        "SM8750-AB (8 Elite for Galaxy)"
+      ]
+    },
+    "81": {
+      "name": "Hexagon V81",
+      "socs": [
+        "SM8850 (Snapdragon 8 Elite Gen 5)"
+      ]
+    }
+  },
+  "io_contracts": {
+    "ours-sdxl-emb": {
+      "graph_name": "model",
+      "unet_inputs": {
+        "sample": "[1,4,128,128] UFIXED16 (NCHW, raw scheduler latent)",
+        "emb": "[1,1280] UFIXED16 host combined-emb = time_embedding(time_proj(t)) + add_embedding(cat(pooled, add_time_proj(time_ids))); computed host-side via sdxl_emb_mlp.bin to fold-proof BOTH sinusoidal paths qairt-converter 2.40 would constant-fold",
+        "text_embedding": "[1,77,2048] UFIXED16 = concat CLIP-L last_hidden[1,77,768] ++ bigG last_hidden[1,77,1280]"
+      },
+      "unet_output": "output [1,4,128,128] UFIXED16",
+      "text_encoder_1": "CLIP-L: in input_ids INT32 [1,77] -> out last_hidden [1,77,768] (penultimate, hidden -2)",
+      "text_encoder_2": "OpenCLIP-bigG (+projection): in input_ids INT32 [1,77] -> out last_hidden [1,77,1280] + pooled [1,1280] (pooled feeds the host emb compute)",
+      "vae": "TAESDXL CPU tiny-VAE (taesdxl_decoder.mnn), in latent [1,4,128,128] -> out [1,3,1024,1024]; CPU MNN path only (SDXL_VAE_BACKEND=cpu)",
+      "scheduler": "EulerAncestralDiscrete, trailing spacing, epsilon prediction, 4 steps native, guidance_scale 0 (NO CFG -> ONE UNet pass per step)",
+      "vae_scaling_factor": 0.13025,
+      "resolution": 1024,
+      "driver": "libsdxl_qnn_driver.so (#51 ImageGenerationSdxlNpuCliHelper); flags --text_encoder_1/--text_encoder_2/--unet/--vae_mnn/--emb_mlp/--vocab_1/--merges_1/--vocab_2/--merges_2/--backend/--system_library; env SDXL_VAE_BACKEND=cpu"
+    }
+  },
+  "sets": {
+    "ours/v75": {
+      "source": "ours",
+      "dsp_arch": 75,
+      "representative_soc": "SM8650 (Snapdragon 8 Gen 3)",
+      "soc_model": 57,
+      "qairt_runtime_required": "2.40",
+      "skel": "libQnnHtpV75Skel.so",
+      "io_contract": "ours-sdxl-emb",
+      "verified": true,
+      "verify_note": "Device-verified coherent+crisp on Lenovo SM8650/V75 (.69), EulerAncestral, 4-6 steps. Forward-compat verified: these exact V75-native bins also ran coherent+crisp on the RedMagic SM8850/V81 handset (.82) and a second V81 (.80). Native-V81/V79 SDXL bins NOT separately built; V75-native is the shipping set for V73->V81.",
+      "files": {
+        "unet.bin": {
+          "path": "v75/unet.bin",
+          "size": 2637398016,
+          "md5": "d7be73631e7c2a57875517c9af09dbe8"
+        },
+        "text_encoder_1_fp16.bin": {
+          "path": "v75/text_encoder_1_fp16.bin",
+          "size": 234770432,
+          "md5": "4e9612177b6ec68a4c1ea992174cdff7"
+        },
+        "text_encoder_2_fp16.bin": {
+          "path": "v75/text_encoder_2_fp16.bin",
+          "size": 1401405440,
+          "md5": "3aa8a2ad01be2ba854f1a847ec16a9b8"
+        },
+        "sdxl_emb_mlp.bin": {
+          "path": "v75/sdxl_emb_mlp.bin",
+          "size": 29184000,
+          "md5": "0cf25084f04c78560d3e58cde177f28c"
+        },
+        "taesdxl_decoder.mnn": {
+          "path": "v75/taesdxl_decoder.mnn",
+          "size": 2464388,
+          "md5": "bfbfb43faa70fc3885bfef24a60b8882"
+        },
+        "tokenizer/vocab.json": {
+          "path": "v75/tokenizer/vocab.json",
+          "size": 1059962,
+          "md5": "a3933521163a0e76cee6094f1d72a9cc"
+        },
+        "tokenizer/merges.txt": {
+          "path": "v75/tokenizer/merges.txt",
+          "size": 524619,
+          "md5": "d2bc10797e70c920c14c69729ecdab34"
+        },
+        "tokenizer/special_tokens_map.json": {
+          "path": "v75/tokenizer/special_tokens_map.json",
+          "size": 586,
+          "md5": "f1c97d73dc836331f2138893217d32d8"
+        },
+        "tokenizer/tokenizer_config.json": {
+          "path": "v75/tokenizer/tokenizer_config.json",
+          "size": 704,
+          "md5": "b0e83247e6c27802b6f2ccce2615e402"
+        },
+        "tokenizer_2/vocab.json": {
+          "path": "v75/tokenizer_2/vocab.json",
+          "size": 1059962,
+          "md5": "a3933521163a0e76cee6094f1d72a9cc"
+        },
+        "tokenizer_2/merges.txt": {
+          "path": "v75/tokenizer_2/merges.txt",
+          "size": 524619,
+          "md5": "d2bc10797e70c920c14c69729ecdab34"
+        },
+        "tokenizer_2/special_tokens_map.json": {
+          "path": "v75/tokenizer_2/special_tokens_map.json",
+          "size": 460,
+          "md5": "1f2ba80c3ba1db3a1729f82f0c2667ce"
+        },
+        "tokenizer_2/tokenizer_config.json": {
+          "path": "v75/tokenizer_2/tokenizer_config.json",
+          "size": 855,
+          "md5": "7a07b6cea8034258f7a153860e81bc41"
+        },
+        "config.json": {
+          "path": "v75/config.json",
+          "size": 936,
+          "md5": "833a8f609063eee89c093754af978210"
+        }
+      }
+    }
+  },
+  "driver_required_files": [
+    "unet.bin",
+    "text_encoder_1_fp16.bin",
+    "text_encoder_2_fp16.bin",
+    "sdxl_emb_mlp.bin",
+    "taesdxl_decoder.mnn",
+    "tokenizer/vocab.json",
+    "tokenizer/merges.txt",
+    "tokenizer_2/vocab.json",
+    "tokenizer_2/merges.txt"
+  ],
+  "install_marker_files": [
+    "unet.bin",
+    "text_encoder_1_fp16.bin",
+    "text_encoder_2_fp16.bin",
+    "taesdxl_decoder.mnn"
+  ],
+  "bundle_dir_name": "SDXL-QNN-NPU",
+  "model_id": "image-sdxl-qnn-npu",
+  "total_bytes": 4308394979,
+  "total_human": "4.31 GB (4.01 GiB)"
+}

v75/config.json

@@ -0,0 +1,25 @@
+{
+  "model": "sdxl-turbo",
+  "pipeline": "StableDiffusionXLPipeline",
+  "variant": "fp16-encoders + TAESDXL",
+  "dsp_arch": 75,
+  "skel": "libQnnHtpV75Skel.so",
+  "qairt_runtime_required": "2.40",
+  "quantization": "W8A16",
+  "resolution": 1024,
+  "latent": [1, 4, 128, 128],
+  "scheduler": "EulerAncestralDiscrete",
+  "steps_default": 4,
+  "guidance_scale": 0.0,
+  "cross_attention_dim": 2048,
+  "vae_scaling_factor": 0.13025,
+  "text_encoders": {
+    "text_encoder_1": "CLIP-L (penultimate, hidden -2) -> [1,77,768]",
+    "text_encoder_2": "OpenCLIP-bigG (+projection) -> last_hidden [1,77,1280] + pooled [1,1280]"
+  },
+  "text_embedding": "concat CLIP-L[768] ++ bigG[1280] -> [1,77,2048]",
+  "host_emb": "sdxl_emb_mlp.bin -> emb[1,1280] per step (fold-proofs timestep + add_time)",
+  "vae": "TAESDXL CPU tiny-VAE (taesdxl_decoder.mnn) via --vae_mnn, SDXL_VAE_BACKEND=cpu",
+  "graph_name": "model",
+  "io_contract": "ours-sdxl-emb"
+}

v75/sdxl_emb_mlp.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82b3a85d8dc3942aa4e1b55d134180258317a863a46023f075d806fdc4c1184c
+size 29184000

v75/taesdxl_decoder.mnn

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3174bb23f3aadbe59ca365aca59d00b8998a0410364ee859882affe91f10b43
+size 2464388

v75/text_encoder_1_fp16.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7dfe0b318196ae5aa7ce8d592c171c1c057f2382dd7285e7ed0ffb9a1f5c521
+size 234770432

v75/text_encoder_2_fp16.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0a87055f6465fb4887d02a53867dd5a3a00c0b5d479882295b097f1741f68ce
+size 1401405440

v75/tokenizer/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|startoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

v75/tokenizer/tokenizer_config.json

@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "49406": {
+      "content": "<|startoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49407": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|startoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "do_lower_case": true,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 77,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "CLIPTokenizer",
+  "unk_token": "<|endoftext|>"
+}

v75/tokenizer_2/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|startoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "!",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

v75/tokenizer_2/tokenizer_config.json

@@ -0,0 +1,38 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "!",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49406": {
+      "content": "<|startoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49407": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|startoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "do_lower_case": true,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 77,
+  "pad_token": "!",
+  "tokenizer_class": "CLIPTokenizer",
+  "unk_token": "<|endoftext|>"
+}

v75/unet.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2159472360dcd197e6566a97fc85f878ad7b25fd180d64a018df72b9e861b612
+size 2637398016