Instructions to use Noctalin/Ornith-1.0-35B-oQ8-fp16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Noctalin/Ornith-1.0-35B-oQ8-fp16 with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("Noctalin/Ornith-1.0-35B-oQ8-fp16")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use Noctalin/Ornith-1.0-35B-oQ8-fp16 with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Ornith-1.0-35B-oQ8-fp16"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Noctalin/Ornith-1.0-35B-oQ8-fp16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use Noctalin/Ornith-1.0-35B-oQ8-fp16 with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "Noctalin/Ornith-1.0-35B-oQ8-fp16"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "Noctalin/Ornith-1.0-35B-oQ8-fp16"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "Noctalin/Ornith-1.0-35B-oQ8-fp16",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use Noctalin/Ornith-1.0-35B-oQ8-fp16 with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Ornith-1.0-35B-oQ8-fp16"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Noctalin/Ornith-1.0-35B-oQ8-fp16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Noctalin/Ornith-1.0-35B-oQ8-fp16 with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Ornith-1.0-35B-oQ8-fp16"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Noctalin/Ornith-1.0-35B-oQ8-fp16" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Ornith-1.0-35B oQ8 Text-Only (Optimized for Apple Silicon)

This repository contains a custom-quantized, text-only configuration of the Ornith-1.0-35B Mixture-of-Experts (MoE) model, optimized explicitly for local repository-level agentic coding on Apple Silicon using the oMLX inference engine.

🎯 Why This Was Created

Ornith-1.0-35B is a state-of-the-art, self-improving MoE model specialized for agentic coding. It jointly optimizes search scaffolds and solution rollouts via Reinforcement Learning to discover superior code trajectories.

This specific build was converted using oMLX v0.4.5.dev1 to address long-context deployment constraints on a MacBook Pro M2 Max (96GB Unified Memory, 400 GB/s memory bandwidth):

The MoE High-Fidelity Precision: As a 35B Mixture-of-Experts model, Ornith-1.0-35B strikes an incredible balance between deep reasoning and high throughput. Quantizing it to oQ8 (8-bit) provides near-lossless code syntax retention and complex instruction adherence, matching the original model's quality while still saving valuable Unified Memory to support deep context windows during complex multi-file repository tasks.
Overcoming the 128k Context Wall: Traditional backends often choke or suffer severe latency degradation when context sizes scale out. Moving to oMLX's specialized two-tier caching eliminates this overhead, allowing you to fluidly ingest giant codebases.
Prefill Speedup via float16: While the base weights are distributed in bfloat16, this build explicitly targets Apple Silicon hardware by using float16 for non-quantized weights, unlocking a ~20% faster prefill speed on M1/M2 Max chips.
MTP Note: Unlike some Qwen base models, the original Ornith-1.0 weights do not contain Multi-Token Prediction (mtp.*) headers. As a result, native MTP decoding is not available for this model.

🚀 Key Differences

Feature / Attribute	Standard Ornith-1.0-35B	This Custom Build (`oQ8-fp16`)
Native MTP Heads	Not present in base architecture	Not Available (No base MTP tensors)
Vision Model (VLM)	N/A (Text-only coding agent)	Stripped/Text-Only
Quantization Method	Standard Uniform / HF / Unsloth	oQ8 (Dynamic mixed-precision calibration)
Non-Quant Weight DType	`bfloat16`	`float16` (~20% faster prefill on M1/M2 Silicon)

💻 Hardware & RAM Recommendations

Mac Hardware Configuration	RAM Recommendation	Status / Performance Expectation
M1 Max / M2 Max / M3 Max (Base 32GB/36GB/48GB)	48GB Unified Memory	Supported — Fits under 48GB, but leaves limited headroom for massive context windows. 32GB/36GB variations are not recommended.
M1 / M2 / M3 / M4 Max / Ultra	64GB Unified Memory	Recommended — Fits comfortably with ample space for moderate context lengths (~64k-128k).
M2 Max / M3 Max / Ultra (96GB / 128GB / 192GB)	96GB+ Unified Memory	Optimal / Best Experience — Highly recommended configuration. Allows operating at the full context boundary (262k) with zero system memory pressure.

🛠️ Quantization Settings

This model was quantized using oMLX v0.4.5.dev1 with the following specification:

Source Model: deepreinforce-ai/Ornith-1.0-35B
Sensitivity Model: None
oQ Level: oQ8
Text Only: Enabled
Preserve MTP weights: Disabled (Not present in source architecture)
Non-quant weight dtype: float16

⚙️ Optimized oMLX Settings (v0.4.5)

To seamlessly route this model through agentic development workspaces like OpenCode, apply the following server specifications in your oMLX dashboard:

Model Basic Settings

Reasoning Parser: qwen3 (Isolates the <think> ... </think> blocks securely away from IDE syntax parsers)
Tool Call Parser: qwen3_xml
CTX Window: 262,144
Max Tokens: 32,768
Temperature: 0.6 (Use 1.0 if attempting to perfectly replicate official benchmark environments)
Top P / Top K: 0.95 / 20
Min P: 0
Repetition / Presence Penalty: 1 / 0

Model Advanced Settings

Enabled Thinking: Checked (True)
Chat Template Kwargs: enable_thinking: true, preserve_thinking: true
Native MTP: Unchecked (False)

Resource Management & Cache

Memory Guard: Aggressive (Forces strict macOS memory/swap cleanup cycles)
Hot Cache Limit (RAM): 40GB (Allocated for hyper-speed Unified Memory history)
Cold Cache Limit (SSD): 371GB (Serialized safetensors storage for context overflow handles)
Max Concurrent Requests: 2 (Protects the 400 GB/s bandwidth bus from degradation)
Embedding Batch Size: 32
Chunked Prefill: Enabled (Prevents instantaneous out-of-memory crashes on massive project context ingestion)
Burst Decode: Aggressive (Coalesces tokens for maximized typing speeds)
Initial Cache Blocks: 256
SSE Keepalive Mode: Chunk

🩹 Repair Script (`repair_moe_experts.py`)

The weights in this repository are already fixed and load correctly — you do not need to run this for normal use.

It's included only for anyone who cached an earlier broken download of this repo, or who runs into the same issue when quantizing another MoE model with a similar per-expert weight layout. Symptom: loading fails with an error like:

Received 92160 parameters not in model:
language_model.model.layers.0.mlp.experts.0.down_proj.biases, ...

Cause: Ornith-1.0-35B ships its MoE experts as separate per-expert tensors (mlp.experts.<0-255>.{gate,up,down}_proj), but mlx-lm's qwen3_5_moe loader only understands the fused switch_mlp layout — so an affected build never quantizes them into a loadable shape, and its config.json also carries stale pre-sanitize key names for the per-path quantization overrides.

Usage, if ever needed:

python3 repair_moe_experts.py /path/to/Ornith-1.0-35B-oQ8-fp16

Requires only mlx (any environment with mlx-lm/omlx installed has it). It streams the weights shard-by-shard, stacks the per-expert tensors into switch_mlp (bitwise-identical values — no requantization), and fixes the config.json key names. The repair happens in place: new shards are written alongside the originals, verified bitwise against the source tensors, and only then swapped in — a failure at any point leaves the original model untouched. It needs free disk roughly equal to the model size while running, and is safe to re-run (exits early on an already-repaired model).

🌡️ Thermal Optimization Notice

Sustained execution across massive context windows heavily taxes the Apple Silicon GPU/CPU complexes, causing rapid heat buildup. Because Apple's default fan profiles emphasize near-silent operation, they delay ramping up system fans until thermal throttling has already begun to affect generation tokens-per-second (TPS).

To protect performance integrity during prolonged coding sessions, it is highly recommended to run a custom fan utility to enforce proactive, aggressive cooling curves:

Source Repository: smc-command on Codeberg

Downloads last month: 48

Safetensors

Model size

10B params

Tensor type

F16

U32

BF16

MLX

Hardware compatibility

8-bit

Model tree for Noctalin/Ornith-1.0-35B-oQ8-fp16

Base model

ornith-ai/Ornith-1.0-35B

Quantized

(185)

this model

Ornith-1.0-35B oQ8 Text-Only (Optimized for Apple Silicon)

🎯 Why This Was Created

🚀 Key Differences

💻 Hardware & RAM Recommendations

🛠️ Quantization Settings

⚙️ Optimized oMLX Settings (v0.4.5)

Model Basic Settings

Model Advanced Settings

Resource Management & Cache

🩹 Repair Script (repair_moe_experts.py)

🌡️ Thermal Optimization Notice

Model tree for Noctalin/Ornith-1.0-35B-oQ8-fp16

🩹 Repair Script (`repair_moe_experts.py`)