Instructions to use Jeethu/North-Micro-Vision-Instruct-PARO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Jeethu/North-Micro-Vision-Instruct-PARO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Jeethu/North-Micro-Vision-Instruct-PARO")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Jeethu/North-Micro-Vision-Instruct-PARO")
model = AutoModelForMultimodalLM.from_pretrained("Jeethu/North-Micro-Vision-Instruct-PARO", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Jeethu/North-Micro-Vision-Instruct-PARO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Jeethu/North-Micro-Vision-Instruct-PARO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jeethu/North-Micro-Vision-Instruct-PARO",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Jeethu/North-Micro-Vision-Instruct-PARO

SGLang

How to use Jeethu/North-Micro-Vision-Instruct-PARO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Jeethu/North-Micro-Vision-Instruct-PARO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jeethu/North-Micro-Vision-Instruct-PARO",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Jeethu/North-Micro-Vision-Instruct-PARO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jeethu/North-Micro-Vision-Instruct-PARO",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Jeethu/North-Micro-Vision-Instruct-PARO with Docker Model Runner:
```
docker model run hf.co/Jeethu/North-Micro-Vision-Instruct-PARO
```

Jeethu/North-Micro-Vision-Instruct

Pairwise Rotation Quantization for Efficient Reasoning LLM Inference

ParoQuant is the state-of-the-art INT4 quantization for LLMs. It closes the accuracy gap with FP16 while running at near-AWQ speed. Supports NVIDIA GPUs (vLLM, Transformers) and Apple Silicon (MLX). For more information, see https://github.com/z-lab/paroquant.

Jeethu/North-Micro-Vision-Instruct is a 4-bit CohereLabs/North-Micro-Vision-Instruct quantized with ParoQuant.

Evaluation

The following evaluations were run on 2026-08-14 with deterministic greedy decoding. The source checkpoint was evaluated in its native BF16 dtype. This ParoQuant checkpoint uses INT4 language projections (group size 128, krot=8) with FP16 retained tensors, including the vision encoder. For context, the published MLX affine 4-bit checkpoint uses group size 64 with BF16 retained tensors.

Vision

Benchmark	Samples	Source BF16	ParoQuant INT4 / FP16	MLX affine 4-bit / BF16
ChartQA relaxed accuracy	100	81.00%	81.00%	81.00%
MMStar accuracy	1,500	50.53%	50.33%	51.07%

ChartQA uses a fixed seed-0 stratified sample of 50 human_test and 50 augmented_test examples. Scoring follows the VLMEvalKit relaxed rule: case-insensitive exact text matching or a 5% relative tolerance for numeric answers. MMStar uses the complete validation split and extracts the selected option from deterministic generations.

Against source BF16, ParoQuant changes ChartQA by 0.00 percentage points (paired bootstrap 95% CI: -3.00 to +3.00) and MMStar by -0.20 points (95% CI: -1.80 to +1.40). A targeted multi-image color-ordering, synthetic OCR, and object-counting smoke suite was also passed exactly by source BF16 and ParoQuant. MLX was semantically correct on all three cases and exact on two; its color response was verbose.

Text

Benchmark	Samples / tokens	Source BF16	ParoQuant INT4 / FP16	MLX affine 4-bit / BF16
WikiText-2 perplexity (lower is better)	32,704 tokens	30.882	31.106	33.506
ARC-Challenge accuracy	1,172	73.21%	70.56%	69.88%
HellaSwag accuracy	2,000	49.50%	48.20%	49.00%

WikiText-2 perplexity uses 64 non-overlapping sequences of 512 tokens from the test split. ARC-Challenge uses the complete labeled test split. HellaSwag uses a fixed seed-0 sample from the validation split. ARC-Challenge and HellaSwag are zero-shot greedy chat multiple-choice evaluations with identical prompts and tokenization across backends; they are not canonical lm-eval log-likelihood scores.

Against source BF16, the ParoQuant ARC-Challenge delta is -2.65 percentage points (paired bootstrap 95% CI: -4.18 to -1.11), while its HellaSwag delta is -1.30 points (95% CI: -2.80 to +0.20).

Dataset revisions were pinned to 9e63b7df1592a1c2158e735cc1725454aef0d6d9 (ChartQA), bc98d668301da7b14f648724866e57302778ab27 (MMStar), 210d026faf9955653af8916fad021475a3f00453 (ARC), 218ec52e09a7e7462a5400043bb9a69a41d06b76 (HellaSwag), and b08601e04326c79dfdd32d625aee71d232d685c3 (WikiText).