--- license: apache-2.0 base_model: Qwen/Qwen3-4B base_model_relation: quantized pipeline_tag: text-generation library_name: coreai language: - en - zh widget: - text: "Explain on-device AI in one sentence." - text: "Write a haiku about Apple Silicon." tags: - coreai - core-ai - coreai-fabric - aimodel - coreml - apple - apple-silicon - on-device - iphone - metal - text-generation - llm - int8 --- > **Canonical:** [`kevinqz/Qwen3-4B-CoreAI`](https://huggingface.co/kevinqz/Qwen3-4B-CoreAI) — source of truth. # Qwen3-4B (fabric, int8) **Apple Core AI chat model — runs fully on-device on Apple Silicon (iPhone / iPad / Mac, macOS/iOS 27+).** A quantized **stateful KV-cache chat** `.aimodel` — an Apple Core AI conversion of [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B), with an embedded tokenizer + chat template. Produced by [coreai-fabric](https://github.com/kevinqz/coreai-fabric/blob/main/recipes/qwen3-4b-int8.yaml) and indexed by [coreai-catalog](https://github.com/kevinqz/coreai-catalog). ## Model facts | Field | Value | |---|---| | Parameters | 4B | | Architecture | transformer | | Capabilities | text-generation | | Quantization / precision | int8 / float16 | | Context length | 40960 | | On-disk size | 4.0 GB | | Asset kind | stateful KV-cache chat bundle; embedded tokenizer + chat template | | assetVersion | 2.0 | ## Use it Install via the catalog, then run it with Apple's Foundation Models runtime: ```bash pip install coreai-catalog && coreai-catalog install qwen3-4b-int8 ``` ```swift import CoreAILanguageModels import FoundationModels // modelURL = the installed macos/ bundle directory for this model let model = try await CoreAILanguageModel(resourcesAt: modelURL) let session = LanguageModelSession(model: model) let reply = try await session.respond(to: "Explain on-device AI in one sentence.") print(reply) ``` A complete, buildable example lives at [coreai-catalog/examples/llm-chat](https://github.com/kevinqz/coreai-catalog/tree/main/examples/llm-chat). ## Requirements - **Deployment: macOS 27.0+ / iOS 27.0+, Xcode 27+.** The asset serializes with `minimum_os v27`, so the on-device Swift runtime requires macOS/iOS 27+. - A Mac on **macOS 26 can convert and inspect** the asset but **cannot run** it on-device (the Swift runtime needs the 27 SDK). - Apple Silicon. ## Intended use & limitations - **Intended use:** general on-device chat / text generation. Inherits the base model's capabilities, languages, and biases. - **Limitations:** int8 quantized — the high-fidelity tier. Fidelity is **measured, not assumed**: see the Evaluation section for the token-margin greedy fidelity vs the fp16 reference. "Near-lossless" is not claimed — it reports argmax agreement, not a task-quality guarantee. See the Evaluation section for the measured greedy fidelity vs the fp16 reference. ## Evaluation (parity) - **Gate A (structure): passed** — the bundle's layout + metadata were validated on real hardware (Apple Silicon); the asset loads and generates. - **Gate B — greedy fidelity vs the fp16 reference: 94.5% margin-gated (95% CI 89.1–97.3%)** · 93.0% exact-argmax · 100.0% top-5, over 128 teacher-forced tokens, measured on-device (Apple Silicon, macOS 26). Margin-gated forgives near-tie flips (where even the reference flips on rounding noise). This is *fidelity to the reference*, not a quality verdict. Reproduce with `coreai-fabric verify` + the parity runner (`parity-report.json`). - Sample — prompt `The capital of France is` → asset: `Paris. The capital of Germany is Berlin. The capital of Italy is Rome.` - **Runtime throughput (tok/s):** to be published once measured on the on-device (macOS/iOS 27) Swift runtime. Not estimated — real numbers or none. ## Provenance | Field | Value | |---|---| | Base model | [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) @ `1cfa9a7208912126459214e8b04321603b3df60c` | | Converted by | `coreai.llm.export` (version not reported) | | Recipe | [qwen3-4b-int8](https://github.com/kevinqz/coreai-fabric/blob/main/recipes/qwen3-4b-int8.yaml) (recipe_source: fabric) | | Precision / quantization | float16 / int8 | | Conversion date | 2026-07-04 | Machine-readable, in this repo: [`parity-report.json`](./parity-report.json) (gate results) · [`reproduce-manifest.json`](./reproduce-manifest.json) (exact tool + stack + pinned revision to reproduce this conversion) · [`LICENSE`](./LICENSE) (upstream terms). ## License and attribution Weights © 2024 Alibaba Cloud, licensed **apache-2.0** — see the bundled `LICENSE`. This artifact is a **converted + quantized derivative** of the base model (the Apache-2.0 §4(b) change notice): weights were converted to Apple Core AI format and quantized to int8. The conversion itself is community work. ## Links - **Base model:** [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) - **Reproduce:** [recipe `qwen3-4b-int8`](https://github.com/kevinqz/coreai-fabric/blob/main/recipes/qwen3-4b-int8.yaml) · [runnable example](https://github.com/kevinqz/coreai-catalog/tree/main/examples/llm-chat) - **Index:** [coreai-catalog](https://github.com/kevinqz/coreai-catalog) — the neutral registry that ties upstream ↔ this asset ↔ mirror together - [HF Collection](https://huggingface.co/collections/kevinqz/coreai-apple-on-device-6a4879f21c7e1a87c99bcf5a) ## The on-device Core AI ecosystem This conversion is part of a broader open ecosystem for running models on Apple's on-device stack — useful references if you're building here: - [coreai-fabric](https://github.com/kevinqz/coreai-fabric) — the reproducible recipe → `.aimodel` pipeline that produced this asset. - [coreai-catalog](https://github.com/kevinqz/coreai-catalog) — the index of Core AI models across the community, with provenance and integration snippets. - [apple/coreai-models](https://github.com/apple/coreai-models) — Apple's official exporters and runtimes. - [CoreAI Model Zoo](https://github.com/john-rocky/coreai-model-zoo) and the wider [coreai-community](https://huggingface.co/coreai-community) — community conversions across many model families. ## Not affiliated with Apple Community conversion. Not produced, hosted, or endorsed by Apple. Apple and Core AI are trademarks of Apple Inc., used here only to describe the target runtime/format. This is an independent community conversion.