Instructions to use Noctalin/Ornith-1.0-35B-oQ5-fp16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Noctalin/Ornith-1.0-35B-oQ5-fp16 with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("Noctalin/Ornith-1.0-35B-oQ5-fp16")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use Noctalin/Ornith-1.0-35B-oQ5-fp16 with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Ornith-1.0-35B-oQ5-fp16"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Noctalin/Ornith-1.0-35B-oQ5-fp16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use Noctalin/Ornith-1.0-35B-oQ5-fp16 with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "Noctalin/Ornith-1.0-35B-oQ5-fp16"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "Noctalin/Ornith-1.0-35B-oQ5-fp16"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "Noctalin/Ornith-1.0-35B-oQ5-fp16",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use Noctalin/Ornith-1.0-35B-oQ5-fp16 with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Ornith-1.0-35B-oQ5-fp16"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Noctalin/Ornith-1.0-35B-oQ5-fp16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Noctalin/Ornith-1.0-35B-oQ5-fp16 with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Ornith-1.0-35B-oQ5-fp16"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Noctalin/Ornith-1.0-35B-oQ5-fp16" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Ornith-1.0-35B oQ5 Text-Only (Optimized for Apple Silicon)

This repository contains a custom-quantized, text-only configuration of the Ornith-1.0-35B Mixture-of-Experts (MoE) model, optimized explicitly for local repository-level agentic coding on Apple Silicon using the oMLX inference engine.

🎯 Why This Was Created

Ornith-1.0-35B is a state-of-the-art, self-improving MoE model specialized for agentic coding. It jointly optimizes search scaffolds and solution rollouts via Reinforcement Learning to discover superior code trajectories.

This specific build was converted using oMLX v0.4.5.dev1 to address long-context deployment constraints on a MacBook Pro M2 Max (96GB Unified Memory, 400 GB/s memory bandwidth):

The MoE Precision Balance: As a 35B Mixture-of-Experts model, Ornith-1.0-35B strikes an incredible balance between deep reasoning and high throughput. Quantizing it to oQ5 (5-bit) delivers a notable step up in code syntax retention and complex instruction adherence over 4-bit alternatives, while keeping the active footprint low enough to maximize headroom for deep KV caches during repository-wide multi-file edits.
Overcoming the 128k Context Wall: Traditional backends often choke or suffer severe latency degradation when context windows approach 128k tokens due to unoptimized KV cache processing. Moving to oMLX's specialized two-tier caching eliminates this overhead, allowing you to ingest expansive codebases fluidly.
Prefill Speedup via float16: While the base weights are distributed in bfloat16, this build explicitly targets Apple Silicon hardware by using float16 for non-quantized weights, unlocking a ~20% faster prefill speed on M1/M2 Max architectures.
MTP Note: Unlike standard Qwen base models, the original Ornith-1.0 weights do not contain Multi-Token Prediction (mtp.*) headers. As a result, native MTP decoding is not available for this model layout.

🚀 Key Differences

Feature / Attribute	Standard Ornith-1.0-35B	This Custom Build (`oQ5-fp16`)
Native MTP Heads	Not present in base architecture	Not Available (No base MTP tensors)
Vision Model (VLM)	N/A (Text-only coding agent)	Stripped/Text-Only
Quantization Method	Standard Uniform / HF / Unsloth	oQ5 (Dynamic mixed-precision calibration)
Non-Quant Weight DType	`bfloat16`	`float16` (~20% faster prefill on M1/M2 Silicon)

💻 Hardware & RAM Recommendations

Mac Hardware Configuration	RAM Recommendation	Status / Performance Expectation
M1 Max / M2 Max / M3 Max (Base 32GB/36GB/48GB)	48GB Unified Memory	Supported — Fits nicely due to the lean MoE memory footprint. 32GB/36GB variations can deploy it but will face tighter constraints on extreme context scaling.
M1 / M2 / M3 / M4 Max / Ultra	64GB Unified Memory	Recommended — Great overall execution, mapping deep context lengths (~128k) comfortably.
M2 Max / M3 Max (96GB / 128GB)	96GB / 128GB Unified Memory	Optimal / Best Experience — The 96GB/128GB setup is ideal. Allows running maximum context extensions (262k) with zero slowdowns and substantial system memory overhead.

🛠️ Quantization Settings

This model was quantized using oMLX v0.4.5.dev1 with the following configuration:

Source Model: deepreinforce-ai/Ornith-1.0-35B
Sensitivity Model: None
oQ Level: oQ5
Text Only: Enabled
Preserve MTP weights: Disabled (Not present in source architecture)
Non-quant weight dtype: float16

⚙️ Optimized oMLX Settings (v0.4.5)

To replicate a highly stable workspace inside coding environments like OpenCode, utilize these server parameters in your oMLX dashboard:

Model Basic Settings

Reasoning Parser: qwen3 (Isolates the <think> ... </think> block safely away from standard IDE syntax highlights)
Tool Call Parser: qwen3_xml
CTX Window: 262,144
Max Tokens: 32,768
Temperature: 0.6 (Apply 1.0 if you want to explicitly match the official benchmark configuration)
Top P / Top K: 0.95 / 20
Min P: 0
Repetition / Presence Penalty: 1 / 0

Model Advanced Settings

Enabled Thinking: Checked (True)
Chat Template Kwargs: enable_thinking: true, preserve_thinking: true
Native MTP: Unchecked (False)

Resource Management & Cache

Memory Guard: Aggressive (Enforces strict macOS memory/swap garbage cleanup)
Hot Cache Limit (RAM): 40GB (Allocated for high-speed Unified Memory history)
Cold Cache Limit (SSD): 371GB (Serialized context overflow storage protection)
Max Concurrent Requests: 2 (Prevents dividing the 400 GB/s bandwidth bus unnecessarily)
Embedding Batch Size: 32
Chunked Prefill: Enabled (Eliminates localized memory spikes during giant codebase file indexing)
Burst Decode: Aggressive (Coalesces tokens for swift user-facing code rendering)
Initial Cache Blocks: 256
SSE Keepalive Mode: Chunk

🩹 Repair Script (`repair_moe_experts.py`)

The weights in this repository are already fixed and load correctly — you do not need to run this for normal use.

It's included only for anyone who cached an earlier broken download of this repo, or who runs into the same issue when quantizing another MoE model with a similar per-expert weight layout. Symptom: loading fails with an error like:

Received 92160 parameters not in model:
language_model.model.layers.0.mlp.experts.0.down_proj.biases, ...

Cause: Ornith-1.0-35B ships its MoE experts as separate per-expert tensors (mlp.experts.<0-255>.{gate,up,down}_proj), but mlx-lm's qwen3_5_moe loader only understands the fused switch_mlp layout — so an affected build never quantizes them into a loadable shape, and its config.json also carries stale pre-sanitize key names for the per-path quantization overrides.

Usage, if ever needed:

python3 repair_moe_experts.py /path/to/Ornith-1.0-35B-oQ5-fp16

Requires only mlx (any environment with mlx-lm/omlx installed has it). It streams the weights shard-by-shard, stacks the per-expert tensors into switch_mlp (bitwise-identical values — no requantization), and fixes the config.json key names. The repair happens in place: new shards are written alongside the originals, verified bitwise against the source tensors, and only then swapped in — a failure at any point leaves the original model untouched. It needs free disk roughly equal to the model size while running, and is safe to re-run (exits early on an already-repaired model).

🌡️ Thermal Optimization Notice

Sustained execution over deep contexts heavily loads the Apple Silicon SoC, raising internal core temperatures. Because native macOS fan curves prioritize absolute quiet over proactive temperature maintenance, they often delay full fan deployment until minor thermal throttling occurs.

To prevent generational speed degradation during deep code agent sessions, run a specialized CLI fan control package to manage system thermals directly:

Source Repository: smc-command on Codeberg

Downloads last month: 61

Safetensors

Model size

7B params

Tensor type

F16

U32

BF16

MLX

Hardware compatibility

5-bit

Model tree for Noctalin/Ornith-1.0-35B-oQ5-fp16

Base model

ornith-ai/Ornith-1.0-35B

Quantized

(185)

this model

Collection including Noctalin/Ornith-1.0-35B-oQ5-fp16

Ornith-1.0 oQ (M1/M2 Apple Silicon)

Collection

Text-only Ornith-1.0 oQ models optimized for M1/M2 Apple Silicon. Built for local agentic coding via oMLX and OpenCode. • 2 items • Updated Jul 3

Ornith-1.0-35B oQ5 Text-Only (Optimized for Apple Silicon)

🎯 Why This Was Created

🚀 Key Differences

💻 Hardware & RAM Recommendations

🛠️ Quantization Settings

⚙️ Optimized oMLX Settings (v0.4.5)

Model Basic Settings

Model Advanced Settings

Resource Management & Cache

🩹 Repair Script (repair_moe_experts.py)

🌡️ Thermal Optimization Notice

Model tree for Noctalin/Ornith-1.0-35B-oQ5-fp16

Collection including Noctalin/Ornith-1.0-35B-oQ5-fp16

🩹 Repair Script (`repair_moe_experts.py`)