kevinqz/Qwen3-4B-CoreAI

Canonical: kevinqz/Qwen3-4B-CoreAI — source of truth.

Qwen3-4B (fabric, int8)

Apple Core AI chat model — runs fully on-device on Apple Silicon (iPhone / iPad / Mac, macOS/iOS 27+).

A quantized stateful KV-cache chat .aimodel — an Apple Core AI conversion of Qwen/Qwen3-4B, with an embedded tokenizer + chat template. Produced by coreai-fabric and indexed by coreai-catalog.

Model facts

Field	Value
Parameters	4B
Architecture	transformer
Capabilities	text-generation
Quantization / precision	int8 / float16
Context length	40960
On-disk size	4.0 GB
Asset kind	stateful KV-cache chat bundle; embedded tokenizer + chat template
assetVersion	2.0

Use it

Install via the catalog, then run it with Apple's Foundation Models runtime:

pip install coreai-catalog && coreai-catalog install qwen3-4b-int8

import CoreAILanguageModels
import FoundationModels

// modelURL = the installed macos/ bundle directory for this model
let model = try await CoreAILanguageModel(resourcesAt: modelURL)
let session = LanguageModelSession(model: model)
let reply = try await session.respond(to: "Explain on-device AI in one sentence.")
print(reply)

A complete, buildable example lives at coreai-catalog/examples/llm-chat.

Requirements

• Deployment: macOS 27.0+ / iOS 27.0+, Xcode 27+. The asset serializes with minimum_os v27, so the on-device Swift runtime requires macOS/iOS 27+.
• A Mac on macOS 26 can convert and inspect the asset but cannot run it on-device (the Swift runtime needs the 27 SDK).
• Apple Silicon.

Intended use & limitations

• Intended use: general on-device chat / text generation. Inherits the base model's capabilities, languages, and biases.
• Limitations: int8 quantized — the high-fidelity tier. Fidelity is measured, not assumed: see the Evaluation section for the token-margin greedy fidelity vs the fp16 reference. "Near-lossless" is not claimed — it reports argmax agreement, not a task-quality guarantee. See the Evaluation section for the measured greedy fidelity vs the fp16 reference.

Evaluation (parity)

• Gate A (structure): passed — the bundle's layout + metadata were validated on real hardware (Apple Silicon); the asset loads and generates.
• Gate B — greedy fidelity vs the fp16 reference: 94.5% margin-gated (95% CI 89.1–97.3%) · 93.0% exact-argmax · 100.0% top-5, over 128 teacher-forced tokens, measured on-device (Apple Silicon, macOS 26). Margin-gated forgives near-tie flips (where even the reference flips on rounding noise). This is fidelity to the reference, not a quality verdict. Reproduce with coreai-fabric verify + the parity runner (parity-report.json).
  • Sample — prompt The capital of France is → asset: Paris. The capital of Germany is Berlin. The capital of Italy is Rome.
• Runtime throughput (tok/s): to be published once measured on the on-device (macOS/iOS 27) Swift runtime. Not estimated — real numbers or none.

Provenance

Field	Value
Base model	Qwen/Qwen3-4B @ 1cfa9a7208912126459214e8b04321603b3df60c
Converted by	coreai.llm.export (version not reported)
Recipe	qwen3-4b-int8 (recipe_source: fabric)
Precision / quantization	float16 / int8
Conversion date	2026-07-04

Machine-readable, in this repo: parity-report.json (gate results) · reproduce-manifest.json (exact tool + stack + pinned revision to reproduce this conversion) · LICENSE (upstream terms).

License and attribution

Weights © 2024 Alibaba Cloud, licensed apache-2.0 — see the bundled LICENSE. This artifact is a converted + quantized derivative of the base model (the Apache-2.0 §4(b) change notice): weights were converted to Apple Core AI format and quantized to int8. The conversion itself is community work.

Links

• Base model: Qwen/Qwen3-4B
• Reproduce: recipe qwen3-4b-int8 · runnable example
• Index: coreai-catalog — the neutral registry that ties upstream ↔ this asset ↔ mirror together
• HF Collection

The on-device Core AI ecosystem

This conversion is part of a broader open ecosystem for running models on Apple's on-device stack — useful references if you're building here:

• coreai-fabric — the reproducible recipe → .aimodel pipeline that produced this asset.
• coreai-catalog — the index of Core AI models across the community, with provenance and integration snippets.
• apple/coreai-models — Apple's official exporters and runtimes.
• CoreAI Model Zoo and the wider coreai-community — community conversions across many model families.

Not affiliated with Apple

Community conversion. Not produced, hosted, or endorsed by Apple. Apple and Core AI are trademarks of Apple Inc., used here only to describe the target runtime/format. This is an independent community conversion.