Instructions to use mlboydaisuke/North-Micro-Vision-CoreAI with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mlboydaisuke/North-Micro-Vision-CoreAI with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="mlboydaisuke/North-Micro-Vision-CoreAI")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mlboydaisuke/North-Micro-Vision-CoreAI", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mlboydaisuke/North-Micro-Vision-CoreAI with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mlboydaisuke/North-Micro-Vision-CoreAI"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mlboydaisuke/North-Micro-Vision-CoreAI",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/mlboydaisuke/North-Micro-Vision-CoreAI

SGLang

How to use mlboydaisuke/North-Micro-Vision-CoreAI with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mlboydaisuke/North-Micro-Vision-CoreAI" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mlboydaisuke/North-Micro-Vision-CoreAI",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mlboydaisuke/North-Micro-Vision-CoreAI" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mlboydaisuke/North-Micro-Vision-CoreAI",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use mlboydaisuke/North-Micro-Vision-CoreAI with Docker Model Runner:
```
docker model run hf.co/mlboydaisuke/North-Micro-Vision-CoreAI
```

North-Micro-Vision-Instruct — Apple Core AI (`.aimodel`)

Cohere's 2.4B multilingual VLM converted to Apple's Core AI (the Core ML successor announced at WWDC26), running on iOS 27 and macOS 27. On an iPhone 17 Pro it answers about an image at 18.2 tok/s and 24/24 tokens identical to the fp32 reference — not "close", identical.

Two bundles, run in sequence: a 400M vision tower (custom-trained from SigLIP2-SO400M, and structurally a Qwen3-VL visual encoder — deepstack mergers included) emitting image_embeds [256, 2048] + deepstack_embeds [768, 2048], and a 2B Cohere decoder with the image tokens spliced in as extension ids. Eleven languages, including Japanese. Apache-2.0.

Requires iOS 27 / macOS 27 (Core AI ships with the OS). Conversion code, gates and knowledge base: coreai-model-zoo.

Bundles

path	size	measured	numerics
`gpu-pipelined/north_micro_vision_instruct_vision_fp16`	1.0 GB	83.4 ms/image (M4 Max)	`image_embeds` cos 0.999996 vs fp32
`gpu-pipelined/north_micro_vision_instruct_decode_int8lin`	2.4 GB	—	suite 9/9 cases, 338/338 tokens
`gpu-pipelined/…_decode_int8lin_textcore`	2.4 GB	145.3 prompt / 118.6 decode tok/s (M4 Max)	the same weights with no image inputs
`ios-h18p/…_decode_int8lin` + `ios-h18p/…_vision_fp16`	2.5 GB	21.5 prefill / 18.2 decode tok/s (iPhone 17 Pro)	nat 16/16 + image oracle 24/24

M4 Max, macOS 27.0 (26A5378n), Xcode 27.0 (27A5218g), coreai-torch 0.4.1, llm-benchmark -p 128 -g 256 -n 3, COREAI_CHUNK_THRESHOLD=1. The Mac tok/s row is the text core because llm-runner cannot bind an image buffer. The iPhone rows are PipelinedBench, including a mandatory 1024-token generation (the iOS compiler miscompiles KV specializations at seq ≥ 2048 and a 256-token run cannot see it) — clean.

int8 costs this model nothing measurable. int4 is not published: 0 of 9 cases, with a lost sentence boundary and instruction boilerplate leaking in ("…a blanket.Answer: Cats.I apologize, but I cannot provide a detailed description of the image") plus flat repetition. int4 tolerance is a property of the individual model — a sibling port at 3B took int4 for free and a 450M one cratered — so read the generations rather than the parameter count.

Run it

git clone https://github.com/apple/coreai-models   # + the zoo's engine patches, see below
swift build -c release --product llm-runner

COREAI_CHUNK_THRESHOLD=1 .build/release/llm-runner \
  --model gpu-pipelined/north_micro_vision_instruct_decode_int8lin_textcore \
  --prompt "The alphabet begins A, B, C," \
  --max-tokens 64 --sampling-strategy greedy \
  --inference-engine-variant coreai-pipelined --warmup off

The coreai-pipelined-static-inputs patch (which binds image_embeds, deepstack_embeds and the two rope-shift scalars) is in the zoo under apps/.

For the image path the host resizes to a 512×512 canvas with an antialiased BICUBIC filter, normalizes (x/255 − 0.5)/0.5, and patchifies into 16×16 patches in Qwen-VL order — block-major over 2×2 merge groups, and [C][T][py][px] inside each patch with the still frame duplicated (patch_dim 1536). Then it runs the tower, binds both outputs, and rewrites the prompt's <image> ids (255031) to V + slot. Reference implementation: _smoke/northmv_preprocess.py.

Note the export bakes a square grid while the upstream processor is native-resolution (it keeps a 640×480 image's own 30×40 patch grid), so non-square images are stretched.

Converting this yourself

The oracle needs transformers git main: the 5.15.0 release does not know cohere_compass and raises on AutoConfig.

The decoder is where the work is, and each of these runs fine when done wrong: a parallel block (one LayerNorm, attention and MLP summed into the residual), Cohere LayerNorm (the mean is subtracted, no bias), SSSF × 7 layer types where the 7 full-attention layers have no positional encoding at all while the 21 sliding ones carry interleaved M-RoPE in a 4096 window, logit_scale 0.25, and a 262 144-entry embedding tied to the head.

The tower needed no work: it loads into the zoo's existing Qwen3-VL encoder with zero missing keys and matches every seam at cos 1.000000.

Everything is in conversion/export_northmv_pipelined.py and knowledge/north-micro-vision-port.md.

License

Apache-2.0, carried from CohereLabs/North-Micro-Vision-Instruct (revision 373bda96ac70bf89f99f7048f420cf00dc07c149). Not affiliated with Apple or Cohere.

Downloads last month: -

Inference Providers NEW

Image-Text-to-Text

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for mlboydaisuke/North-Micro-Vision-CoreAI

Base model

CohereLabs/North-Micro-Vision-Instruct

Finetuned

(5)