Upload README.md with huggingface_hub

README.md

@@ -16,24 +16,33 @@ Apple **Core AI** for iOS 27 / macOS 27 (beta), riding Apple's **`coreai-pipelin
 GPU engine** via the decode-only loop-free export — async encode, on-GPU argmax
 sampling, on-device KV growth, zero custom kernels.
 
-| surface | prefill (S=1) | decode |
+| surface (ship bundle) | prefill (S=1) | decode |
 |---|---:|---:|
-| **M4 Max** (release `llm-benchmark`, p=128 g=256) | 127.0 | **127.2 tok/s** |
-| iPhone 17 Pro (one-shot runner) | 19.6–22.4 | 19.1–21.3 tok/s |
+| **M4 Max** (release `llm-benchmark`, p=128 g=256) | 161.2 | **160.8 tok/s** |
+| **iPhone 17 Pro** (one-shot runner, 2 runs × 2 trials) | 29.7–30.3 | **28–30 tok/s** — ≥ the CoreML qwen3.5-2B port (~27) |
 
 Numerics: **16/16 teacher-forced single-step top-1 vs the fp32 HF oracle + HF-cache-seeded
-decode step** (the [zoo](https://github.com/john-rocky/coreai-model-zoo) ship gate), and the
-iPhone greedy sequences are **24/24 token-identical to the Mac GPU** on both fixed prompts.
+decode step** (the [zoo](https://github.com/john-rocky/coreai-model-zoo) ship gate), greedy
+rollouts token-identical to the fp16-head bundle, and the iPhone sequences are **24/24
+token-identical to the Mac GPU** on both fixed prompts.
 
-## Bundle
+## Bundles
 
-`gpu-pipelined/qwen3_5_2b_decode_int8lin/` — full LanguageBundle (`metadata.json` +
-`tokenizer/` + `.aimodel`), **int8 linear per-block-32** (no LUT — 256-entry LUT dequant is
-slow on the GPU delegate), fp16 embed + tied lm_head in-graph, 2.39 GB.
-`input_ids` is STATIC `[1,1]` (loop-free single-step GDN ≡ the scan at S=1); position_ids +
-KV seq stay dynamic, so `EngineFactory` classifies it dynamic → pipelined engine.
+- **`gpu-pipelined/qwen3_5_2b_decode_int8hu_perchan_sym/` — the ship config (2.9 GB)**:
+  transformer int8 linear per-block-32 + **untied lm_head in per-channel absmax int8**
+  (`int8hu --head-quant perchan --head-sym`). The head trick is what unlocks the speed: the
+  248 K-vocab fp16 head was ~1.0 GB of the ~2.4 GB per-token read. Crucial detail: the head
+  must be quantized with plain **absmax `symmetric`** — the default
+  `symmetric_with_clipping` clips outlier head rows and flips oracle top-1s (full story in
+  the zoo's [pipelined-engine notes](https://github.com/john-rocky/coreai-model-zoo/blob/main/knowledge/pipelined-engine.md)).
+- `gpu-pipelined/qwen3_5_2b_decode_int8lin/` — fp16-head variant (2.4 GB): 127 tok/s Mac /
+  19–21 iPhone. Smaller; keep if you want the head at full precision.
 
-## Run (macOS — recommended surface)
+Both are full LanguageBundles (`metadata.json` + `tokenizer/` + `.aimodel`), `input_ids`
+STATIC `[1,1]` (loop-free single-step GDN), position_ids + KV seq dynamic → `EngineFactory`
+classifies them dynamic → pipelined engine.
+
+## Run (macOS)
 
 Needs the engine patch stack from the
 [zoo](https://github.com/john-rocky/coreai-model-zoo) (`apps/coreai-shared-product.patch` →
@@ -41,7 +50,7 @@ Needs the engine patch stack from the
 as patches), then:
 
 ```bash
-COREAI_CHUNK_THRESHOLD=1 llm-benchmark --model qwen3_5_2b_decode_int8lin -p 128 -g 256 -n 3
+COREAI_CHUNK_THRESHOLD=1 llm-benchmark --model qwen3_5_2b_decode_int8hu_perchan_sym -p 128 -g 256 -n 3
 ```
 
 - `COREAI_CHUNK_THRESHOLD=1` **before engine creation** — prefill runs as pipelined S=1 steps
@@ -51,23 +60,23 @@ COREAI_CHUNK_THRESHOLD=1 llm-benchmark --model qwen3_5_2b_decode_int8lin -p 128
   `--warmup exact --warmup-length 1`).
 - Benchmark **Release** builds only (a Debug engine measures ~3× slow).
 
-## iPhone (runs, but not the fastest phone option)
+## iPhone
 
-The same bundle runs on iPhone 17 Pro at 19–21 tok/s with exact numerics. Know before you ship:
+The ship bundle decodes 28–30 tok/s on iPhone 17 Pro with exact numerics. Know before you ship:
 
 - Requires the **`com.apple.developer.kernel.increased-memory-limit`** entitlement — cold GPU
   specialization dies with `std::bad_alloc` at the default jetsam limit without it.
-- Cold specialization 29.1 s (then 3.0 s warm loads, content-keyed cache). Keep **≥4 GB free
-  disk**: the spec cache is ~3.5 GB, and a failed cold spec leaves partial caches that make
+- Cold specialization 22.3 s (then ~5.6 s warm loads, content-keyed cache). Keep **≥4 GB free
+  disk**: the spec cache is ~3 GB, and a failed cold spec leaves partial caches that make
   later attempts fail with `NSPOSIXErrorDomain code=2` at engine create — uninstall the app to
   reclaim.
-- Decode is fully bandwidth-bound (~2.4 GB weight read/token); the CoreML Qwen3.5-2B port
-  (~27 tok/s) is still the faster phone option, and the
-  [0.8B pipelined bundle](https://huggingface.co/mlboydaisuke/qwen3.5-0.8B-CoreAI) does 50+.
+- For smaller phones / tighter RAM, the
+  [0.8B pipelined bundle](https://huggingface.co/mlboydaisuke/qwen3.5-0.8B-CoreAI) does 50+
+  tok/s in 1 GB.
 
 ## Reproduce
 
 Conversion script (self-contained) + method page in the zoo:
 [`conversion/export_qwen3_5_decode_pipelined.py`](https://github.com/john-rocky/coreai-model-zoo/blob/main/conversion/export_qwen3_5_decode_pipelined.py)
-(`int8lin --hf-id Qwen/Qwen3.5-2B`) ·
+(`int8hu --head-quant perchan --head-sym --hf-id Qwen/Qwen3.5-2B`) ·
 [`knowledge/pipelined-engine.md`](https://github.com/john-rocky/coreai-model-zoo/blob/main/knowledge/pipelined-engine.md)