--- license: mit license_link: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B/blob/main/LICENSE base_model: - deepreinforce-ai/Ornith-1.0-35B pipeline_tag: text-generation tags: - mlx - code - text-only - omlx - ornith - ornith-1.0 - ornith-35B - MoE library_name: mlx --- # Ornith-1.0-35B oQ4 Text-Only (Optimized for Apple Silicon) This repository contains a custom-quantized, text-only configuration of the **Ornith-1.0-35B** Mixture-of-Experts (MoE) model, optimized explicitly for local repository-level agentic coding on Apple Silicon using the **oMLX** inference engine. ## 🎯 Why This Was Created Ornith-1.0-35B is a state-of-the-art, self-improving MoE model specialized for agentic coding. It jointly optimizes search scaffolds and solution rollouts via Reinforcement Learning to discover superior code trajectories. This specific build was converted using **oMLX v0.4.5.dev1** to address long-context deployment constraints on a **MacBook Pro M2 Max (96GB Unified Memory, 400 GB/s memory bandwidth)**: * **The MoE Memory Optimization:** As a 35B Mixture-of-Experts model, Ornith-1.0-35B strikes an incredible balance between deep reasoning and high throughput. Quantizing it to **oQ4 (4-bit)** dramatically lowers its initial RAM footprint. This leaves maximum VRAM headroom dedicated entirely to holding massive KV caches during complex multi-file repository tasks. * **Overcoming the 128k Context Wall:** Traditional backends often choke or suffer severe latency degradation when context sizes scale out. Moving to oMLX's specialized two-tier caching eliminates this overhead, allowing you to fluidly ingest giant codebases. * **Prefill Speedup via float16:** While the base weights are distributed in `bfloat16`, this build explicitly targets Apple Silicon hardware by using **`float16`** for non-quantized weights, unlocking a **~20% faster prefill speed** on M1/M2 Max chips. * **MTP Note:** Unlike some Qwen base models, the original Ornith-1.0 weights do not contain Multi-Token Prediction (`mtp.*`) headers. As a result, native MTP decoding is not available for this model. --- ## 🚀 Key Differences | Feature / Attribute | Standard Ornith-1.0-35B | This Custom Build (`oQ4-fp16`) | | :--- | :--- | :--- | | **Native MTP Heads** | Not present in base architecture | **Not Available** (No base MTP tensors) | | **Vision Model (VLM)** | N/A (Text-only coding agent) | **Stripped/Text-Only** | | **Quantization Method** | Standard Uniform / HF / Unsloth | **oQ4** (Dynamic mixed-precision calibration) | | **Non-Quant Weight DType**| `bfloat16` | **`float16`** (~20% faster prefill on M1/M2 Silicon) | --- ## 💻 Hardware & RAM Recommendations | Mac Hardware Configuration | RAM Recommendation | Status / Performance Expectation | | :--- | :--- | :--- | | **M1 Max / M2 Max / M3 Max (Base 32GB/36GB/48GB)** | **32GB/48GB Unified Memory** | **Supported** — Fits comfortably due to the lean 4-bit MoE footprint. Leaves reasonable space for mid-sized context windows. | | **M1 / M2 / M3 / M4 Max / Ultra** | **64GB Unified Memory** | **Recommended** — Great overall performance, comfortably maps deep context lengths (~128k). | | **M2 Max / M3 Max (96GB / 128GB)** | **96GB / 128GB Unified Memory** | **Optimal / Best Experience** — Ideal configuration for a 96GB M2 Max. Allows operating at the full 262k context boundary with zero system memory pressure. | --- ## 🛠️ Quantization Settings This model was quantized using **oMLX v0.4.5.dev1** with the following specification: * **Source Model:** `deepreinforce-ai/Ornith-1.0-35B` * **Sensitivity Model:** `None` * **oQ Level:** `oQ4` * **Text Only:** `Enabled` * **Preserve MTP weights:** `Disabled` (Not present in source architecture) * **Non-quant weight dtype:** `float16` --- ## ⚙️ Optimized oMLX Settings (v0.4.5) To seamlessly route this model through agentic development workspaces like **OpenCode**, apply the following server specifications in your oMLX dashboard: ### Model Basic Settings * **Reasoning Parser:** `qwen3` *(Isolates the ` ... ` blocks securely away from IDE syntax parsers)* * **Tool Call Parser:** `qwen3_xml` * **CTX Window:** `262,144` * **Max Tokens:** `32,768` * **Temperature:** `0.6` *(Use `1.0` if attempting to perfectly replicate official benchmark environments)* * **Top P / Top K:** `0.95 / 20` * **Min P:** `0` * **Repetition / Presence Penalty:** `1 / 0` ### Model Advanced Settings * **Enabled Thinking:** Checked (True) * **Chat Template Kwargs:** `enable_thinking: true, preserve_thinking: true` * **Native MTP:** Unchecked (False) ### Resource Management & Cache * **Memory Guard:** `Aggressive` (Forces strict macOS memory/swap cleanup cycles) * **Hot Cache Limit (RAM):** `40GB` (Allocated for hyper-speed Unified Memory history) * **Cold Cache Limit (SSD):** `371GB` (Serialized safetensors storage for context overflow handles) * **Max Concurrent Requests:** `2` (Protects the 400 GB/s bandwidth bus from degradation) * **Embedding Batch Size:** `32` * **Chunked Prefill:** Enabled (Prevents instantaneous out-of-memory crashes on massive project context ingestion) * **Burst Decode:** `Aggressive` (Coalesces tokens for maximized typing speeds) * **Initial Cache Blocks:** `256` * **SSE Keepalive Mode:** `Chunk` --- ## 🩹 Repair Script (`repair_moe_experts.py`) The weights in this repository are already fixed and load correctly — **you do not need to run this for normal use.** It's included only for anyone who cached an earlier broken download of this repo, or who runs into the same issue when quantizing another MoE model with a similar per-expert weight layout. Symptom: loading fails with an error like: ``` Received 92160 parameters not in model: language_model.model.layers.0.mlp.experts.0.down_proj.biases, ... ``` Cause: Ornith-1.0-35B ships its MoE experts as separate per-expert tensors (`mlp.experts.<0-255>.{gate,up,down}_proj`), but mlx-lm's `qwen3_5_moe` loader only understands the fused `switch_mlp` layout — so an affected build never quantizes them into a loadable shape, and its `config.json` also carries stale pre-sanitize key names for the per-path quantization overrides. Usage, if ever needed: ```bash python3 repair_moe_experts.py /path/to/Ornith-1.0-35B-oQ4-fp16 ``` Requires only `mlx` (any environment with `mlx-lm`/`omlx` installed has it). It streams the weights shard-by-shard, stacks the per-expert tensors into `switch_mlp` (bitwise-identical values — no requantization), and fixes the `config.json` key names. The repair happens **in place**: new shards are written alongside the originals, verified bitwise against the source tensors, and only then swapped in — a failure at any point leaves the original model untouched. It needs free disk roughly equal to the model size while running, and is safe to re-run (exits early on an already-repaired model). ## 🌡️ Thermal Optimization Notice Sustained execution across massive context windows heavily taxes the Apple Silicon GPU/CPU complexes, causing rapid heat buildup. Because Apple's default fan profiles emphasize near-silent operation, they delay ramping up system fans until thermal throttling has already begun to affect generation tokens-per-second (TPS). To protect performance integrity during prolonged coding sessions, it is highly recommended to run a custom fan utility to enforce proactive, aggressive cooling curves: * Source Repository: [smc-command on Codeberg](https://codeberg.org/Noctalin/smc-command)