---
license: mit
license_link: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B/blob/main/LICENSE
base_model:
- deepreinforce-ai/Ornith-1.0-35B
pipeline_tag: text-generation
tags:
- mlx
- code
- text-only
- omlx
- ornith
- ornith-1.0
- ornith-35B
- MoE
library_name: mlx
---

# Ornith-1.0-35B oQ5 Text-Only (Optimized for Apple Silicon)

This repository contains a custom-quantized, text-only configuration of the **Ornith-1.0-35B** Mixture-of-Experts (MoE) model, optimized explicitly for local repository-level agentic coding on Apple Silicon using the **oMLX** inference engine.

## 🎯 Why This Was Created

Ornith-1.0-35B is a state-of-the-art, self-improving MoE model specialized for agentic coding. It jointly optimizes search scaffolds and solution rollouts via Reinforcement Learning to discover superior code trajectories. 

This specific build was converted using **oMLX v0.4.5.dev1** to address long-context deployment constraints on a **MacBook Pro M2 Max (96GB Unified Memory, 400 GB/s memory bandwidth)**:

* **The MoE Precision Balance:** As a 35B Mixture-of-Experts model, Ornith-1.0-35B strikes an incredible balance between deep reasoning and high throughput. Quantizing it to **oQ5 (5-bit)** delivers a notable step up in code syntax retention and complex instruction adherence over 4-bit alternatives, while keeping the active footprint low enough to maximize headroom for deep KV caches during repository-wide multi-file edits.
* **Overcoming the 128k Context Wall:** Traditional backends often choke or suffer severe latency degradation when context windows approach 128k tokens due to unoptimized KV cache processing. Moving to oMLX's specialized two-tier caching eliminates this overhead, allowing you to ingest expansive codebases fluidly.
* **Prefill Speedup via float16:** While the base weights are distributed in `bfloat16`, this build explicitly targets Apple Silicon hardware by using **`float16`** for non-quantized weights, unlocking a **~20% faster prefill speed** on M1/M2 Max architectures.
* **MTP Note:** Unlike standard Qwen base models, the original Ornith-1.0 weights do not contain Multi-Token Prediction (`mtp.*`) headers. As a result, native MTP decoding is not available for this model layout.

---

## 🚀 Key Differences

| Feature / Attribute | Standard Ornith-1.0-35B | This Custom Build (`oQ5-fp16`) |
| :--- | :--- | :--- |
| **Native MTP Heads** | Not present in base architecture | **Not Available** (No base MTP tensors) |
| **Vision Model (VLM)** | N/A (Text-only coding agent) | **Stripped/Text-Only** |
| **Quantization Method** | Standard Uniform / HF / Unsloth | **oQ5** (Dynamic mixed-precision calibration) |
| **Non-Quant Weight DType**| `bfloat16` | **`float16`** (~20% faster prefill on M1/M2 Silicon) |

---

## 💻 Hardware & RAM Recommendations

| Mac Hardware Configuration | RAM Recommendation | Status / Performance Expectation |
| :--- | :--- | :--- |
| **M1 Max / M2 Max / M3 Max (Base 32GB/36GB/48GB)** | **48GB Unified Memory** | **Supported** — Fits nicely due to the lean MoE memory footprint. 32GB/36GB variations can deploy it but will face tighter constraints on extreme context scaling. |
| **M1 / M2 / M3 / M4 Max / Ultra** | **64GB Unified Memory** | **Recommended** — Great overall execution, mapping deep context lengths (~128k) comfortably. |
| **M2 Max / M3 Max (96GB / 128GB)** | **96GB / 128GB Unified Memory** | **Optimal / Best Experience** — The 96GB/128GB setup is ideal. Allows running maximum context extensions (262k) with zero slowdowns and substantial system memory overhead. |

---

## 🛠️ Quantization Settings

This model was quantized using **oMLX v0.4.5.dev1** with the following configuration:

* **Source Model:** `deepreinforce-ai/Ornith-1.0-35B`
* **Sensitivity Model:** `None`
* **oQ Level:** `oQ5`
* **Text Only:** `Enabled`
* **Preserve MTP weights:** `Disabled` (Not present in source architecture)
* **Non-quant weight dtype:** `float16`

---

## ⚙️ Optimized oMLX Settings (v0.4.5)

To replicate a highly stable workspace inside coding environments like **OpenCode**, utilize these server parameters in your oMLX dashboard:

### Model Basic Settings
* **Reasoning Parser:** `qwen3` *(Isolates the `<think> ... </think>` block safely away from standard IDE syntax highlights)*
* **Tool Call Parser:** `qwen3_xml`
* **CTX Window:** `262,144`
* **Max Tokens:** `32,768`
* **Temperature:** `0.6` *(Apply `1.0` if you want to explicitly match the official benchmark configuration)*
* **Top P / Top K:** `0.95 / 20`
* **Min P:** `0`
* **Repetition / Presence Penalty:** `1 / 0`

### Model Advanced Settings
* **Enabled Thinking:** Checked (True)
* **Chat Template Kwargs:** `enable_thinking: true, preserve_thinking: true`
* **Native MTP:** Unchecked (False)

### Resource Management & Cache
* **Memory Guard:** `Aggressive` (Enforces strict macOS memory/swap garbage cleanup)
* **Hot Cache Limit (RAM):** `40GB` (Allocated for high-speed Unified Memory history)
* **Cold Cache Limit (SSD):** `371GB` (Serialized context overflow storage protection)
* **Max Concurrent Requests:** `2` (Prevents dividing the 400 GB/s bandwidth bus unnecessarily)
* **Embedding Batch Size:** `32`
* **Chunked Prefill:** Enabled (Eliminates localized memory spikes during giant codebase file indexing)
* **Burst Decode:** `Aggressive` (Coalesces tokens for swift user-facing code rendering)
* **Initial Cache Blocks:** `256`
* **SSE Keepalive Mode:** `Chunk`

---

## 🩹 Repair Script (`repair_moe_experts.py`)

The weights in this repository are already fixed and load correctly — **you do not need to run this for normal use.**

It's included only for anyone who cached an earlier broken download of this repo, or who runs into the same issue when quantizing another MoE model with a similar per-expert weight layout. Symptom: loading fails with an error like:

```
Received 92160 parameters not in model:
language_model.model.layers.0.mlp.experts.0.down_proj.biases, ...
```

Cause: Ornith-1.0-35B ships its MoE experts as separate per-expert tensors (`mlp.experts.<0-255>.{gate,up,down}_proj`), but mlx-lm's `qwen3_5_moe` loader only understands the fused `switch_mlp` layout — so an affected build never quantizes them into a loadable shape, and its `config.json` also carries stale pre-sanitize key names for the per-path quantization overrides.

Usage, if ever needed:

```bash
python3 repair_moe_experts.py /path/to/Ornith-1.0-35B-oQ5-fp16
```

Requires only `mlx` (any environment with `mlx-lm`/`omlx` installed has it). It streams the weights shard-by-shard, stacks the per-expert tensors into `switch_mlp` (bitwise-identical values — no requantization), and fixes the `config.json` key names. The repair happens **in place**: new shards are written alongside the originals, verified bitwise against the source tensors, and only then swapped in — a failure at any point leaves the original model untouched. It needs free disk roughly equal to the model size while running, and is safe to re-run (exits early on an already-repaired model).

## 🌡️ Thermal Optimization Notice

Sustained execution over deep contexts heavily loads the Apple Silicon SoC, raising internal core temperatures. Because native macOS fan curves prioritize absolute quiet over proactive temperature maintenance, they often delay full fan deployment until minor thermal throttling occurs.

To prevent generational speed degradation during deep code agent sessions, run a specialized CLI fan control package to manage system thermals directly:

* Source Repository: [smc-command on Codeberg](https://codeberg.org/Noctalin/smc-command)