---
license: mit
base_model: deepreinforce-ai/Ornith-1.0-35B
base_model_relation: quantized
pipeline_tag: text-generation
library_name: mlx-vlm
tags:
  - mlx
  - mlx-vlm
  - mlx-lm
  - omlx
  - oq
  - oq5
  - quantized
  - qwen3_5_moe
  - moe
  - vision-language
  - image-text-to-text
  - agentic-coding
  - safetensors
---

# Ornith-1.0-35B-oQ5

This repository contains an unofficial oMLX oQ5 MLX quantization of
[`deepreinforce-ai/Ornith-1.0-35B`](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B).

The source model is a Qwen3.5 MoE vision-language model released by DeepReinforce
under the MIT license. This quantized artifact is intended for Apple Silicon
inference with oMLX / MLX-compatible runtimes.

## Quantization Summary

- Source model: `deepreinforce-ai/Ornith-1.0-35B`
- Output format: MLX safetensors
- Quantizer: oMLX oQ streaming quantization
- oQ level: `oQ5`
- Base quantization: 5-bit affine
- Group size: 64
- Storage dtype for non-quantized floating tensors: BF16
- Output size: about 24 GB on disk
- Output shards: 6 safetensors shards
- Per-layer quantization overrides: 352
- Vision weights: preserved
- MTP/speculative head: disabled in the output config because the source
  checkpoint does not contain `mtp.*` tensors

The final `config.json` contains both `quantization` and
`quantization_config` entries so MLX loaders can apply the mixed-precision
layout.

## Why This Is Not a Plain Stock oQ Run

This checkpoint required two compatibility workarounds during quantization.

First, the source `config.json` advertises an MTP head:

```json
"mtp_num_hidden_layers": 1
```

but the downloaded checkpoint does not contain corresponding `mtp.*` tensors.
That causes the stock oMLX auto-proxy sensitivity path to fail before it can
measure calibration sensitivity. For the quantized output, MTP was normalized to
disabled (`0`) so the config matches the actual weights.

Second, the source checkpoint uses VLM-prefixed Qwen3.5 MoE expert keys. During
the first streaming quantization pass, the backbone MoE experts were emitted as
individual per-expert tensors such as:

```text
language_model.model.layers.N.mlp.experts.E.gate_proj.weight
language_model.model.layers.N.mlp.experts.E.up_proj.weight
language_model.model.layers.N.mlp.experts.E.down_proj.weight
```

The current MLX/VLM loader expects those backbone expert weights in fused
`switch_mlp` form:

```text
language_model.model.layers.N.mlp.switch_mlp.gate_proj.weight
language_model.model.layers.N.mlp.switch_mlp.up_proj.weight
language_model.model.layers.N.mlp.switch_mlp.down_proj.weight
```

After quantization, the per-expert quantized tensors were stacked into the
loader-compatible `switch_mlp` layout. This repack is structural: it stacks the
already quantized `weight`, `scales`, and `biases` tensors. It does not
dequantize, requantize, or otherwise change the quantized numeric values.

## Sensitivity Allocation

oQ normally measures layer sensitivity by running calibration inference and then
allocating higher precision to the layers where quantization error matters most.
For this checkpoint, the automatic proxy calibration path was not usable for the
reasons above. Instead, quantization used an explicit 40-layer positional
sensitivity map:

- first and last 12.5% of layers: highest sensitivity
- next outer quartiles: moderate sensitivity
- middle layers: lower sensitivity

That means this is still mixed-precision oQ5, but it is not a fully
data-calibrated oQ artifact. The expected implication is that quality should be
reasonable for an oQ5 quantization, but the bit allocation is less tailored than
a successful calibration-based oQ run.

## Conversion Procedure

The final artifact was produced locally from the original BF16 safetensors
checkpoint using the oMLX app bundle's Python environment.

High-level steps:

1. Created a temporary symlinked source view of the original model.
2. Set stale MTP config fields to `0` in that temporary view only.
3. Added an explicit `oq_sensitivity_map.json` to bypass the incompatible
   auto-proxy sensitivity path.
4. Ran oMLX `quantize_oq_streaming` with:
   - `oq_level=5`
   - `group_size=64`
   - `dtype="bfloat16"`
   - `text_only=False`
   - `preserve_mtp=False`
   - `auto_proxy_sensitivity=False`
5. Repacked quantized MoE expert tensors from per-expert keys into
   `switch_mlp` keys expected by the current MLX/VLM loader.
6. Validated the final directory with a lazy MLX/VLM load.

The source model directory was not modified.

## Final Validation

The final model directory passed these checks after repacking:

```text
Output size:              24G
Safetensors shards:       6
Quantization bits:        5
Quantization group size:  64
Quantization mode:        affine
Per-layer overrides:      352
Mapped tensors:           2010
switch_mlp tensors:       360
per-expert MoE keys:      0
MTP layer count:          0
Lazy VLM load:            passed
```

Lazy load validation used the oMLX app bundle with `mlx_vlm.utils.load_model`
and `mlx_lm.tokenizer_utils.load`. This validates the config/weight-key layout
and catches loader mismatches. It is not the same as a full generation benchmark.

## Usage

This model is meant to be used with oMLX or another runtime that supports MLX
safetensors quantization metadata. After downloading or cloning the repository
into an oMLX model directory, load it as a normal local MLX model.

For oMLX CLI usage with a local model directory:

```bash
omlx serve --model-dir /path/to/models
```

Then select or request the model by its directory/repository name, depending on
your oMLX setup.

The original Ornith model card notes that Ornith is a reasoning model and may
produce a `<think>...</think>` block before the final answer. Preserve the
original chat template included with this repository when serving the model.

## Known Limitations

- This is an unofficial community quantization, not an official DeepReinforce
  release.
- The oQ sensitivity map was heuristic/positional rather than measured by
  calibration inference, because the stock oMLX auto-proxy path could not load
  this checkpoint cleanly.
- Native MTP/speculative decoding is disabled because the source checkpoint did
  not contain MTP weights despite advertising an MTP layer in config.
- Lazy load validation passed, but no benchmark suite was run as part of this
  conversion.
- Text and vision weights are present, but image-generation or image-question
  workflows should be smoke-tested in the target runtime before relying on this
  upload for production VLM use.

## Relationship to the Original Model

Please cite and refer to the original model for architecture details, intended
use, benchmark results, license, and upstream caveats:

- Original model: <https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B>
- Organization: DeepReinforce
- License: MIT

This repository only changes the storage/quantization format for MLX inference.
It does not introduce additional training or fine-tuning.