Instructions to use cjnielson44/gpt-oss-120b-oQ4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cjnielson44/gpt-oss-120b-oQ4 with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("cjnielson44/gpt-oss-120b-oQ4")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use cjnielson44/gpt-oss-120b-oQ4 with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "cjnielson44/gpt-oss-120b-oQ4"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "cjnielson44/gpt-oss-120b-oQ4"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use cjnielson44/gpt-oss-120b-oQ4 with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "cjnielson44/gpt-oss-120b-oQ4"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default cjnielson44/gpt-oss-120b-oQ4

Run Hermes

hermes

OpenClaw new

How to use cjnielson44/gpt-oss-120b-oQ4 with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "cjnielson44/gpt-oss-120b-oQ4"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "cjnielson44/gpt-oss-120b-oQ4" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MLX LM

How to use cjnielson44/gpt-oss-120b-oQ4 with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "cjnielson44/gpt-oss-120b-oQ4"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "cjnielson44/gpt-oss-120b-oQ4"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "cjnielson44/gpt-oss-120b-oQ4",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

gpt-oss-120b-oQ4

cjnielson44/gpt-oss-120b-oQ4 is an Apple Silicon / oMLX-ready MLX checkpoint for GPT-OSS 120B. It was produced with oMLX oQ4 quantization and published for local inference through oMLX.

This checkpoint is not a uniform 4-bit conversion of every tensor. GPT-OSS uses MoE expert projection tensors that are already stored in MXFP4 form, so those expert tensors are preserved as MXFP4 passthrough tensors and explicitly marked in config.json.

Quantization Details

Source: local oMLX-compatible gpt-oss-120b MLX checkpoint, originally derived from openai/gpt-oss-120b.
Quantizer: oMLX oQ4.
Main quantized tensors: affine oQ4, bits: 4, group_size: 64.
GPT-OSS MoE expert projections: MXFP4 passthrough, bits: 4, group_size: 32, mode: mxfp4.
Expert override coverage: all 36 layers for gate_proj, up_proj, and down_proj under model.layers.<i>.mlp.experts.
Floating dtype used during quantization: bfloat16.

The MXFP4 expert overrides are required for oMLX/MLX loading. Without them, the loader treats the expert tensors as affine-quantized tensors and expects *.biases parameters that do not exist for these MXFP4 expert weights.

Use With oMLX

Download into an oMLX-discoverable model directory:

hf download cjnielson44/gpt-oss-120b-oQ4 \
  --local-dir ~/.omlx/models/cjnielson44/gpt-oss-120b-oQ4

Restart oMLX, then use this model id:

gpt-oss-120b-oQ4

Example OpenAI-compatible request, assuming your oMLX server is listening locally:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OMLX_API_KEY" \
  -d '{
    "model": "gpt-oss-120b-oQ4",
    "messages": [{"role": "user", "content": "Write a short note about Apple Silicon inference."}],
    "max_tokens": 128
  }'

Choosing This Variant

Use oQ4 if you want the smallest non-expert tensor precision among these releases. Because GPT-OSS expert tensors are preserved as MXFP4 in all three variants, the practical disk-size difference between oQ4, oQ6, and oQ8 may be smaller than expected.

For best quality from this release family, prefer cjnielson44/gpt-oss-120b-oQ8.

Verification

This repo was uploaded after local oMLX discovery and load/unload smoke testing. The same GPT-OSS MXFP4 expert override fix used for oQ8 was applied to this oQ4 repo.

Limitations

Experimental community quantization.
Requires recent oMLX/MLX support for GPT-OSS and MXFP4 expert tensors.
No benchmark or perplexity numbers are provided yet.
This model card does not change the upstream license or usage terms of openai/gpt-oss-120b.

Downloads last month: 523

Safetensors

Model size

22B params

Tensor type

BF16

U32

MLX

Hardware compatibility

4-bit

Model tree for cjnielson44/gpt-oss-120b-oQ4

Base model

openai/gpt-oss-120b

Quantized

(115)

this model