Ornith-1.0-35B-UD-Q2_K_XL-MLX

An MLX quantization of deepreinforce-ai/Ornith-1.0-35B for running locally on Apple Silicon — the default model for chad, a Claude-Code-style local coding agent.

Quant

• Params: 35B (MoE, 256 experts / top-8, ~3B active per token)
• Scheme: 2-bit experts with 3-bit expert down-projections / 6-bit backbone / 8-bit router (3.09 bits per weight)
• Footprint: ~13.4 GB weights, ~67 tok/s decode on a 24 GB M4 Pro
• RAM: Best on ≥24 GB. Does not fit 16/18 GB Macs — chad uses the 9B there.

The naming follows Unsloth's Dynamic 2.0 convention (UD-Q2_K_XL = a dynamic quant that spends extra bits on the layers that matter) so the scheme is recognizable at a glance. It is not literally a llama.cpp k-quant — this is an MLX group-64 affine quant produced by our own per-module predicate (the bulk low-bit, sensitive layers high), with the bit allocation chosen by measured held-out perplexity per gigabyte across candidate recipes.

Use it

With chad (auto-downloads this model on first run):

uvx --from git+https://github.com/nathansutton/mlxcc chad

Or directly with mlx-lm:

from mlx_lm import load, generate
model, tok = load("nathansutton/Ornith-1.0-35B-UD-Q2_K_XL-MLX")
print(generate(model, tok, "Write a haiku about quantization.", max_tokens=64))

License

Inherits the license of the base model deepreinforce-ai/Ornith-1.0-35B. Review it before use or redistribution.