Instructions to use bearzi/GLM-5.2-JANGTQ-mxfp4attn with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bearzi/GLM-5.2-JANGTQ-mxfp4attn with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("bearzi/GLM-5.2-JANGTQ-mxfp4attn")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use bearzi/GLM-5.2-JANGTQ-mxfp4attn with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "bearzi/GLM-5.2-JANGTQ-mxfp4attn"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "bearzi/GLM-5.2-JANGTQ-mxfp4attn"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use bearzi/GLM-5.2-JANGTQ-mxfp4attn with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "bearzi/GLM-5.2-JANGTQ-mxfp4attn"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default bearzi/GLM-5.2-JANGTQ-mxfp4attn

Run Hermes

hermes

OpenClaw new

How to use bearzi/GLM-5.2-JANGTQ-mxfp4attn with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "bearzi/GLM-5.2-JANGTQ-mxfp4attn"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "bearzi/GLM-5.2-JANGTQ-mxfp4attn" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MLX LM

How to use bearzi/GLM-5.2-JANGTQ-mxfp4attn with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "bearzi/GLM-5.2-JANGTQ-mxfp4attn"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "bearzi/GLM-5.2-JANGTQ-mxfp4attn"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "bearzi/GLM-5.2-JANGTQ-mxfp4attn",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

GLM-5.2-JANGTQ-mxfp4attn

bearzi/GLM-5.2-JANGTQ (2-bit routed experts) with its fp16 attention, shared experts, embeddings and lm_head replaced by the 4-bit mxfp4 tensors from mlx-community/GLM-5.2-mxfp4.

Why

The JANGTQ recipe quantizes routed experts aggressively but keeps attention and shared experts at higher precision, since they are read on every token and carry model coherence. In this source bundle they were left at fp16 rather than the 8-bit affine the recipe specifies - roughly 10% of parameters, read in full each token, dominating decode time.

Swapping them for 4-bit mxfp4 cuts that traffic ~4x. The same change applied to GLM-5.2-JANGTQ_K took decode from 10.6 to 18.8 tok/s on an M3 Ultra; this variant has not been separately benchmarked.

Recipe

tensors	format
routed experts	JANGTQ 2-bit (unchanged from source)
attention (`q_a`, `q_b`, `kv_a`, `o_proj`, MLA absorb, indexer)	mxfp4 4-bit
shared experts	mxfp4 4-bit
`embed_tokens`, `lm_head`	mxfp4 4-bit
norms, router gate	fp16

Note mxfp4 ships MLA attention in absorbed form (embed_q / unembed_out), so the source bundle's kv_b_proj tensors are superseded and unreferenced. Attention at 4 bits is below the 8-bit affine the JANGTQ recipe calls for.

Assembly

Built by server-side copy on the Hub - the source shards were duplicated, the mxfp4-bearing shards copied in from bearzi/GLM-5.2-JANGTQ_K-mxfp4attn, and a new model.safetensors.index.json written mapping each tensor to the right shard. Some shards therefore contain tensors the index does not reference (the superseded fp16 attention). A loader that follows the index rather than globbing model-*.safetensors is required.

Requirements

Stock oMLX cannot load JANGTQ glm_moe_dsa checkpoints. Use:

https://github.com/yohann-bearzi/omlx      branch feat/turboquant-steel-moe
https://github.com/yohann-bearzi/jangq     branch glm52-jangtq

built with OMLX_WITH_CUSTOM_KERNEL=1 (Metal toolchain, Python 3.11-3.13).

Validation

Not benchmarked or evaluated. The equivalent change on GLM-5.2-JANGTQ_K was spot-checked for coherence; this bundle has not been. Treat as untested.