Nemotron-3-Embed-1B-oQ2-MLX

An Apple-Silicon-native OMLX oQ2 conversion of nvidia/Nemotron-3-Embed-1B-BF16.

This derivative was produced locally on Apple Silicon with the same streaming, sensitivity-driven OMLX pipeline used for the oQ5 release. It uses a base 2-bit affine plan, group size 64, and bfloat16 working dtype. The calibration ran on the checkpoint's bidirectional encoder architecture rather than through a causal-LM wrapper. No weights were retrained.

The quantized safetensors payload is 427,993,279 bytes. OMLX promoted 23 sensitivity-selected projections to 5, 6-bit precision.

Embedding contract

• bidirectional attention (is_causal: false)
• average/mean pooling
• L2-normalized 2,048-dimensional output
• query: prefix for queries
• passage: prefix for documents

BF16 fidelity smoke test

The source and this oQ2 checkpoint were evaluated using a query, a matching passage, and an unrelated passage. All outputs were 2,048-dimensional and L2-normalized.

Input	BF16 to oQ2 vector cosine
Query	0.934469
Related passage	0.920509
Unrelated passage	0.923551

Comparison	BF16	oQ2
Query to related passage	0.680575	0.715155
Query to unrelated passage	-0.024595	-0.016734

Quick start on Apple Silicon

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade mlx mlx-lm transformers huggingface_hub

hf download groxaxo/Nemotron-3-Embed-1B-oQ2-MLX --local-dir ./Nemotron-3-Embed-1B-oQ2-MLX
python ./Nemotron-3-Embed-1B-oQ2-MLX/embed_mlx.py \
  --model ./Nemotron-3-Embed-1B-oQ2-MLX \
  --query "Which city is known as the City of Sails?" \
  --passage "Auckland is widely known as the City of Sails."

The included script deliberately loads the bare Ministral3 backbone and applies non-causal attention. The regular MLX-LM generation path is causal and is not the correct embedding interface for this checkpoint.

Quantization details

Setting	Value
OMLX source revision	a534507fb5ab760d239283868a4c790386b316ff
Base model revision	a5e0f804b9e90a1ca6784ecbf6e41595774fc834
OMLX level	oQ2
Base quantization	2-bit affine
Group size	64
Working dtype	bfloat16
Calibration	128 samples × 256 tokens
Sensitivity-selected boosts	23

See QUANTIZATION_REPORT.md for artifact SHA-256 values and the full local validation result.

License

This derivative inherits the OpenMDW-1.1 license from the source model. LICENSE, NOTICE, and THIRD_PARTY_NOTICES.md are included unchanged from NVIDIA's source repository.