mlboydaisuke commited on
Commit
25bfd65
·
verified ·
1 Parent(s): 0d7f7f1

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +72 -0
README.md ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: openai/gpt-oss-20b
4
+ tags:
5
+ - coreai
6
+ - apple
7
+ - aimodel
8
+ - apple-silicon
9
+ ---
10
+
11
+ # gpt-oss-20B (MoE) — official Apple Core AI export
12
+
13
+ Pre-converted **`.aimodel` bundles from Apple's official
14
+ [coreai-models](https://github.com/apple/coreai-models) export recipe — unmodified**,
15
+ with the exact environment, hashes, and measured performance published.
16
+
17
+ ```bash
18
+ uv run coreai.llm.export openai/gpt-oss-20b # MXFP4 passes through, ~3 min convert
19
+ ```
20
+
21
+ ## Why pre-converted bundles?
22
+
23
+ 1. **The conversion needs a big-RAM Mac** (the 20B export was done on 128 GB);
24
+ running only needs enough RAM to mmap the artifact.
25
+ 2. **An `.aimodel` is a build artifact, not a pure function of the recipe** — the
26
+ same export command produced a 2.2× slower artifact across the macOS 26 → 27β
27
+ boundary ([forensics](https://github.com/john-rocky/apple-silicon-llm-bench/blob/main/methodology/coreai-export-lowering.md)).
28
+ Hosted artifacts + hashes are the reproducible ground truth; every bundle here
29
+ is exactly the one measured in
30
+ [apple-silicon-llm-bench](https://github.com/john-rocky/apple-silicon-llm-bench).
31
+
32
+ ## Bundles & integrity
33
+
34
+ | Bundle | Contents | SHA-256 (`main.mlirb`) |
35
+ |---|---|---|
36
+ | `macos/` | macOS dynamic, MXFP4 (as shipped by OpenAI) | `63fb96f521f2579efb4d38013037431852bc4f07f471c8b78997ced3d20a230c` |
37
+
38
+ ## Measured (Apple's official `llm-benchmark`, greedy)
39
+
40
+ | Bundle | Protocol | Decode tok/s | Prefill | Load (warm) | Peak RSS |
41
+ |---|---|---:|---:|---:|---:|
42
+ | macos | M4 Max, 512p/1024g | 78.1 | 1,252 | 2.1 s | 33.9 GB |
43
+
44
+ `COREAI_CHUNK_THRESHOLD` is a prefill memory dial on this MoE: unchunked 4096-token
45
+ prefill = 1,439 tok/s at 18 GB dirty footprint; chunk-128 = 766 tok/s at 1.7 GB.
46
+
47
+ ## Export environment
48
+
49
+ - macOS 27.0 beta (build 26A5353q) · Xcode 27.0 (27A5194q)
50
+ - `coreai-core 1.0.0b1` · `coreai-torch 0.4.0` · `coreai-opt 0.2.0` · `torch 2.9.0`
51
+ - apple/coreai-models @ `b1cb71b` (export code identical to upstream `0c1055f`)
52
+
53
+ ## Run it
54
+
55
+ ```bash
56
+ # CLI (from a coreai-models checkout)
57
+ swift run -c release llm-runner --model <downloaded-bundle-dir> --prompt "Hello"
58
+ swift run -c release llm-benchmark --model <downloaded-bundle-dir>
59
+ ```
60
+
61
+ Or chat with it in [CoreAIChatMac](https://github.com/john-rocky/coreai-samples)
62
+ (point "Choose Models Folder…" at the download directory).
63
+
64
+ iOS static bundles must be AOT-compiled before device use:
65
+ `xcrun coreai-build compile <ir>.aimodel --platform iOS --preferred-compute neural-engine --architecture h18p`
66
+ (h18p = iPhone 17 Pro), then set `metadata.json` `assets.main` to the `.aimodelc`.
67
+
68
+
69
+
70
+ ---
71
+ Maintained alongside [coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo)
72
+ (community models) and [coreai-samples](https://github.com/john-rocky/coreai-samples) (apps).