Bonsai-REDUX3-MLX-2bit-pure

Instructions to use jinreiyu/Bonsai-REDUX3-MLX-2bit-pure with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jinreiyu/Bonsai-REDUX3-MLX-2bit-pure with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("jinreiyu/Bonsai-REDUX3-MLX-2bit-pure")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use jinreiyu/Bonsai-REDUX3-MLX-2bit-pure with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "jinreiyu/Bonsai-REDUX3-MLX-2bit-pure"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "jinreiyu/Bonsai-REDUX3-MLX-2bit-pure"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use jinreiyu/Bonsai-REDUX3-MLX-2bit-pure with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "jinreiyu/Bonsai-REDUX3-MLX-2bit-pure"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "jinreiyu/Bonsai-REDUX3-MLX-2bit-pure" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MLX LM

How to use jinreiyu/Bonsai-REDUX3-MLX-2bit-pure with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "jinreiyu/Bonsai-REDUX3-MLX-2bit-pure"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "jinreiyu/Bonsai-REDUX3-MLX-2bit-pure"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "jinreiyu/Bonsai-REDUX3-MLX-2bit-pure",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use jinreiyu/Bonsai-REDUX3-MLX-2bit-pure with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "jinreiyu/Bonsai-REDUX3-MLX-2bit-pure"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default jinreiyu/Bonsai-REDUX3-MLX-2bit-pure

Run Hermes

hermes

Atomic Chat

Bonsai-REDUX3-MLX-2bit-pure / frontier.svg

jinreiyu

Duplicate from jinreiyu/bonsai-2bit-abl-mlx-g5

d2f91f9 3 months ago

preview code

Raw

History Blame Contribute Delete

9.76 kB

	<svg xmlns="http://www.w3.org/2000/svg" width="1500" height="680" viewBox="0 0 1500 680">
	<style>
	.bg { fill: #ede9e4; }
	.title { fill: #2d2a27; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif; font-weight: 700; font-size: 28px; }
	.subtitle { fill: #6a6560; font-family: 'SF Mono', 'Fira Code', 'Fira Mono', 'Roboto Mono', 'Courier New', monospace; font-size: 14px; }
	.separator { stroke: #c5c0ba; }
	.grid { stroke: #d5d0ca; stroke-width: 0.8; }
	.axis { stroke: #2d2a27; stroke-width: 1.2; }
	.tick-label { fill: #6a6560; font-family: 'SF Mono', 'Fira Code', 'Fira Mono', 'Roboto Mono', 'Courier New', monospace; font-size: 13px; }
	.axis-label { fill: #6a6560; font-family: 'SF Mono', 'Fira Code', 'Fira Mono', 'Roboto Mono', 'Courier New', monospace; font-size: 13px; }
	.square-marker { fill: #d4b0b3; fill-opacity: 0.45; stroke: #a07578; stroke-width: 2; }
	.bonsai-fill { fill: #2d2a27; }
	.bonsai-label { fill: #2d2a27; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif; font-weight: 700; font-size: 14px; }
	.prior-fill { fill: #2d2a27; }
	.prior-label { fill: #2d2a27; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif; font-weight: 700; font-size: 14px; }
	.model-label { fill: #8a7072; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif; font-size: 12px; }
	.prior-line { stroke: #c0a8ab; stroke-width: 2; stroke-dasharray: 8 5; fill: none; }
	.bonsai-line { stroke: #2d2a27; stroke-width: 2.5; stroke-dasharray: 8 5; fill: none; }

	@media (prefers-color-scheme: dark) {
	.bg { fill: #1e1c1b; }
	.title { fill: #e8e4e0; }
	.subtitle { fill: #a09890; }
	.separator { stroke: #3a3634; }
	.grid { stroke: #2e2b29; }
	.axis { stroke: #a09890; }
	.tick-label { fill: #a09890; }
	.axis-label { fill: #a09890; }
	.square-marker { fill: #d4b0b3; fill-opacity: 0.35; stroke: #a07578; }
	.bonsai-fill { fill: #e8e4e0; }
	.bonsai-label { fill: #e8e4e0; }
	.prior-fill { fill: #e8e4e0; }
	.prior-label { fill: #e8e4e0; }
	.model-label { fill: #b89498; }
	.prior-line { stroke: #7a6568; }
	.bonsai-line { stroke: #e8e4e0; }
	}
	</style>

	<defs>
	<symbol id="bonsai-icon" viewBox="0 0 24 14.6035">
	<path d="M15.1304 6.25864H12L14.087 10.4311L6.78261 14.6035H16.1739L20.3478 10.4311L15.1304 6.25864Z"/>
	<path d="M16.7826 6.25866H24V8.34488H19.4022L16.7826 6.25866Z"/>
	<path d="M0 6.25866H10.837L11.8804 8.34488H0V6.25866Z"/>
	<path d="M5.21739 3.12933H20.8696V5.21555H5.21739V3.12933Z"/>
	<path d="M10.4348 0H17.7391V2.08622H10.4348V0Z"/>
	</symbol>
	</defs>

	<!-- Background -->
	<rect width="1500" height="680" class="bg" />

	<!-- Title and subtitle -->
	<text x="68" y="48" class="title">Frontier efficiency</text>
	<text x="68" y="72" class="subtitle">Average score (IFEval, GSM8K, HumanEval+, BFCL, MuSR, MMLU-Redux)</text>

	<!-- Separator -->
	<line x1="68" y1="85" x2="1440" y2="85" class="separator" stroke-width="1" />

	<!--
	pixelX(v) = 130 + (ln(v) + 1.8971) / 5.1160 * 1300
	pixelY(v) = 570 - (v - 35) * 9.1

	Ternary: (0.37,57.47)→(359,366) (0.86,70.65)→(574,246) (1.75,75.48)→(755,202)
	1-bit: (0.25,49.30)→(260,440) (0.575,61.6)→(472,328) (1.15,70.5)→(648,247)
	-->

	<!-- Vertical grid lines -->
	<line x1="260" y1="115" x2="260" y2="570" class="grid" />
	<line x1="436" y1="115" x2="436" y2="570" class="grid" />
	<line x1="612" y1="115" x2="612" y2="570" class="grid" />
	<line x1="788" y1="115" x2="788" y2="570" class="grid" />
	<line x1="964" y1="115" x2="964" y2="570" class="grid" />
	<line x1="1141" y1="115" x2="1141" y2="570" class="grid" />
	<line x1="1317" y1="115" x2="1317" y2="570" class="grid" />

	<!-- Axes -->
	<line x1="130" y1="570" x2="1430" y2="570" class="axis" />
	<line x1="130" y1="115" x2="130" y2="570" class="axis" />

	<!-- Y-axis tick labels -->
	<text x="118" y="529" text-anchor="end" class="tick-label">40</text>
	<text x="118" y="438" text-anchor="end" class="tick-label">50</text>
	<text x="118" y="347" text-anchor="end" class="tick-label">60</text>
	<text x="118" y="256" text-anchor="end" class="tick-label">70</text>
	<text x="118" y="165" text-anchor="end" class="tick-label">80</text>

	<!-- Y-axis small ticks -->
	<line x1="125" y1="525" x2="130" y2="525" class="axis" />
	<line x1="125" y1="434" x2="130" y2="434" class="axis" />
	<line x1="125" y1="343" x2="130" y2="343" class="axis" />
	<line x1="125" y1="252" x2="130" y2="252" class="axis" />
	<line x1="125" y1="161" x2="130" y2="161" class="axis" />

	<!-- X-axis tick labels -->
	<text x="260" y="595" text-anchor="middle" class="tick-label">0.25 GB</text>
	<text x="436" y="595" text-anchor="middle" class="tick-label">0.5 GB</text>
	<text x="612" y="595" text-anchor="middle" class="tick-label">1 GB</text>
	<text x="788" y="595" text-anchor="middle" class="tick-label">2 GB</text>
	<text x="964" y="595" text-anchor="middle" class="tick-label">4 GB</text>
	<text x="1141" y="595" text-anchor="middle" class="tick-label">8 GB</text>
	<text x="1317" y="595" text-anchor="middle" class="tick-label">16 GB</text>

	<!-- X-axis small ticks -->
	<line x1="260" y1="570" x2="260" y2="575" class="axis" />
	<line x1="436" y1="570" x2="436" y2="575" class="axis" />
	<line x1="612" y1="570" x2="612" y2="575" class="axis" />
	<line x1="788" y1="570" x2="788" y2="575" class="axis" />
	<line x1="964" y1="570" x2="964" y2="575" class="axis" />
	<line x1="1141" y1="570" x2="1141" y2="575" class="axis" />
	<line x1="1317" y1="570" x2="1317" y2="575" class="axis" />

	<!-- Axis labels -->
	<text x="730" y="635" text-anchor="middle" class="axis-label">Model size in GB (log scale)</text>
	<text x="52" y="343" text-anchor="middle" class="axis-label" transform="rotate(-90, 52, 343)">Average benchmark score</text>

	<!-- Prior Pareto frontier curve (conventional models) -->
	<path d="M 521 572 L 555 541 L 585 515 L 612 492 L 656 456 L 698 424 L 747 389 L 788 361 L 845 326 L 891 300 L 926 282 L 964 263 L 1021 239 L 1067 221 L 1101 210 L 1124 203 L 1142 198 L 1170 191 L 1197 185 L 1221 180 L 1243 176 L 1264 173 L 1283 170 L 1300 168 L 1317 167 L 1323 166 L 1332 166 L 1347 165 L 1360 164 L 1373 164"
	class="prior-line" />

	<!-- Bonsai frontier line (single line through all 6 points) -->
	<path d="M 155 480 L 780 170" class="bonsai-line" />

	<!-- Other model squares -->
	<rect x="650" y="446" width="12" height="12" rx="1" class="square-marker" />
	<rect x="782" y="468" width="12" height="12" rx="1" class="square-marker" />
	<rect x="822" y="457" width="12" height="12" rx="1" class="square-marker" />
	<rect x="836" y="520" width="12" height="12" rx="1" class="square-marker" />
	<rect x="920" y="277" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1079" y="297" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1095" y="216" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1127" y="265" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1136" y="181" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1243" y="326" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1287" y="237" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1298" y="382" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1311" y="236" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1312" y="272" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1312" y="287" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1312" y="367" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1317" y="161" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1320" y="217" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1321" y="249" width="12" height="12" rx="1" class="square-marker" />
	<rect x="1352" y="285" width="12" height="12" rx="1" class="square-marker" />

	<!-- 1-Bit Bonsai icons (faded prior work) -->
	<use href="#bonsai-icon" x="249" y="433" width="22" height="14" class="prior-fill" />
	<use href="#bonsai-icon" x="461" y="321" width="22" height="14" class="prior-fill" />
	<use href="#bonsai-icon" x="637" y="240" width="22" height="14" class="prior-fill" />

	<!-- 1-Bit labels (well below their icons, clear of line) -->
	<text x="180" y="475" class="prior-label">1-Bit Bonsai 1.7B</text>
	<text x="400" y="370" class="prior-label">1-Bit Bonsai 4B</text>
	<text x="660" y="282" class="prior-label">1-Bit Bonsai 8B</text>

	<!-- Ternary Bonsai icons -->
	<use href="#bonsai-icon" x="348" y="359" width="22" height="14" class="bonsai-fill" />
	<use href="#bonsai-icon" x="563" y="239" width="22" height="14" class="bonsai-fill" />
	<use href="#bonsai-icon" x="744" y="195" width="22" height="14" class="bonsai-fill" />

	<!-- Ternary Bonsai labels (well above their icons, clear of line) -->
	<text x="235" y="318" class="bonsai-label">Ternary Bonsai 1.7B</text>
	<text x="445" y="208" class="bonsai-label">Ternary Bonsai 4B</text>
	<text x="655" y="155" class="bonsai-label">Ternary Bonsai 8B</text>

	<!-- Other model labels (frontier models only) -->
	<text x="670" y="445" class="model-label">Qwen3 0.6B</text>
	<text x="940" y="280" class="model-label">Qwen3 1.7B</text>
	<text x="1115" y="215" class="model-label">Ministral3 3B</text>
	<text x="1110" y="175" class="model-label">Qwen3 4B</text>
	<text x="1337" y="160" class="model-label">Qwen3 8B</text>
	</svg>