---
license: other
license_name: minimax-open
library_name: mlx
tags:
- mlx
- jang
- jangtq
- minimax
- moe
- apple-silicon
- 2bit
pipeline_tag: text-generation
base_model: MiniMaxAI/MiniMax-M2.7
base_model_relation: quantized
---
# MiniMax-M2.7 JANGTQ
**MiniMax M2.7 228B MoE — 2.15-bit codebook + Hadamard, 56.5 GB**
### The smallest, highest-quality MiniMax M2.7 on Apple Silicon.
> ⚠️ **Recommended: Run in [MLX Studio](https://mlxstudio.com)** for the best experience. MLX Studio bundles the JANGTQ runtime, handles thinking mode, and uses the custom Metal kernels this model needs. Stock `mlx_lm.load()` will NOT load this model — see usage instructions below.
Follow development on Twitter: **[@jangq_ai](https://twitter.com/jangq_ai)**
---
## What is JANGTQ?
**JANGTQ** (JANG TurboQuant) is the most-compressed, highest-quality JANG
quantization format. Routed expert weights stay in a compact codebook +
Hadamard-rotated form at runtime — no decompression to affine — and the matmul
path uses custom Metal kernels that read packed uint32 weights, look up
centroids in a 4-entry codebook, and accumulate dot products against a
Hadamard-rotated input (QuIP# "rotate-input-once" math).
Result: **smaller than affine 2-bit, higher quality than affine 2-bit, runs at
89% of affine 2-bit speed** on Apple Silicon.
| | JANG_2L (affine) | **JANGTQ** | Δ |
|---|---|---|---|
| Disk size | ~63 GB | **56.5 GB** | **−10%** |
| GPU memory | ~62.6 GB | **56.5 GB** | **−10%** |
| Avg bits/param | 2.10 | **~2.15** | +0.05 |
| MMLU (200q) | 88% | **91.5%** | **+3.5 pp** |
| Decode speed (M3 Ultra) | 48-50 tok/s | **44.3 tok/s** | ~89% of affine |
JANGTQ trades ~10% speed for ~10% disk savings AND a quality improvement.
The 2-bit codebook learned via Lloyd-Max is strictly more expressive than
uniform 2-bit affine for the Gaussian-ish distribution of Hadamard-rotated
weights, so the same bit budget reproduces the original weight matrix more
faithfully.
---
## MMLU Benchmark (200 questions, 10 subjects, reasoning ON)
**Overall: 183/200 = 91.5%**
Tested 2026-04-13 on Mac Studio M3 Ultra. Reasoning enabled (MiniMax M2.7 is
an always-reasoning model); `…` stripped before scoring.
| Subject | JANGTQ | JANG_2L (affine) | JANG_3L/4M |
|---|---|---|---|
| **astronomy** | **20/20 (100%)** | — | — |
| **high_school_biology** | **20/20 (100%)** | — | — |
| abstract_algebra | 19/20 (95%) | — | — |
| college_computer_science | 19/20 (95%) | — | — |
| high_school_mathematics | 19/20 (95%) | — | — |
| college_physics | 18/20 (90%) | — | — |
| high_school_chemistry | 18/20 (90%) | — | — |
| anatomy | 17/20 (85%) | — | — |
| world_religions | 17/20 (85%) | — | — |
| logical_fallacies | 16/20 (80%) | — | — |
| **Total** | **183/200 = 91.5%** | **~88%** | **~95.5%** |
JANGTQ sits cleanly between affine JANG_2L (88%) and the larger JANG_3L/4M
(95.5%) — capturing most of the quality of the 3L/4M profiles at ~55-60% of
their disk footprint.
## Speed Benchmarks (Mac Studio M3 Ultra)
| Prompt / max_tok | observed tok | tok/s |
|---|---|---|
| "Capital of France?" / 50 | 50 / 50 | 35.6 |
| "Capital of France?" / 150 | 66 / 150 | 37.5 |
| "Count 1-30" / 150 | 150 / 150 | 42.2 |
| **"Photosynthesis 5 sent" / 300** | **300 / 300** | **44.5** |
| **"Poem + 17×23" / 300** | **296 / 300** | **44.0** |
| MMLU average (200q, reasoning on) | — | **41.9** |
Steady-state (300-tok and longer): **~44.3 tok/s**. Short prompts appear
slower due to fixed prefill amortization.
---
## Important Settings
MiniMax M2.7 is an **always-reasoning** model. The chat template
unconditionally opens `\n` at each assistant turn.
| Setting | Value | Notes |
|---------|-------|-------|
| Temperature | **1.0** | REQUIRED — temp=0 can cause thinking loops |
| Top P | 0.95 | |
| Top K | 40 | |
| Repetition Penalty | 1.1 | Optional, helps prevent loops |
| max_tokens | ≥ 8192 | Give reasoning room to converge |
Strip `…` from the response before using the final answer.
---
## Model Details
| Metric | Value |
|---|---|
| Source | `MiniMaxAI/MiniMax-M2.7` (FP8 E4M3) |
| Architecture | MoE (256 experts, top-8 active), standard Q/K/V attention, partial RoPE |
| Total parameters | 228.7 B |
| Active per token | ~1.4 B |
| Profile | **JANGTQ** |
| Format | **JANGTQ (codebook+Hadamard)** — `weight_format: mxtq` in `jang_config.json` |
| Avg bits/param | ~2.15 |
| Disk | **56.55 GB** |
| GPU active (loaded) | 56.50 GB |
| GPU peak (decoding) | 57-58 GB |
| Load time | ~10 s |
| Context | 192 K tokens |
| Chat template | Always-reasoning (`\n` opened at assistant start) |
## JANGTQ Bit Allocation
| Component | Bits | Format | Why |
|---|---|---|---|
| **Routed expert MLP** (gate/up/down) — 98% of params | **2** | **JANGTQ codebook + Hadamard** | Sparsely activated (8 of 256 per token); the learned codebook on Hadamard-rotated rows reproduces the distribution better than uniform 2-bit affine |
| Attention (Q/K/V/O) | 8 | affine (`nn.QuantizedLinear`, group_size=64) | Runs on every token; quality-critical |
| Shared expert | 8 | affine | Runs on every token |
| Embed tokens / LM head | 8 | affine | Quality-critical input/output projections |
| Router gate | fp16 | unquantized `nn.Linear` | Routing precision matters; ~0.8M params, negligible size |
| RMSNorms / RoPE / biases | fp16 | unquantized | Already tiny |
The **routed experts** are the 98% of parameters and the natural compression
target. JANGTQ pushes them to 2-bit with a codebook-learned quantizer and a
random Hadamard rotation. Everything else stays at 8-bit affine so the quality-
critical hot path (attention + embed + shared expert) runs at full precision.
---
## Usage
**This model requires the `jang-tools` loader** — stock `mlx_lm.load()` does
NOT recognize `weight_format: mxtq` and will reject the model. The loader
applies Metal kernel monkey-patches at load time (fused gate+up+SwiGLU, gather
TQ, multi-block Hadamard, router compile, QKV fusion, thread-tiling OPT=10/20).
```bash
pip install jang-tools
# Or from source: git clone https://github.com/JANGQ-AI/jang-tools
```
```python
from huggingface_hub import snapshot_download
from jang_tools.load_jangtq import load_jangtq_model
from mlx_lm import generate
model_path = snapshot_download("JANGQ-AI/MiniMax-M2.7-JANGTQ")
model, tokenizer = load_jangtq_model(model_path)
messages = [{"role": "user", "content": "Explain photosynthesis in 5 sentences."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
out = generate(model, tokenizer, prompt, max_tokens=600, verbose=True)
# Strip reasoning to get the final answer
if "" in out:
out = out.split("")[-1].strip()
print(out)
```
On first load you'll see log lines like:
```
Loading JANGTQ: MiniMax-M2.7-JANGTQ
seed=42, bits_map={'attention': 8, ..., 'routed_expert': 2, ...}
61 shards
TQ groups: 47616, regular: 1123
Replaced 186 modules with TurboQuantLinear
Patched SwitchGLU class for fused gate+up (62 TQ instances)
P15 mx.compile(router) applied to 1 MoE class(es)
P18 QKV fusion: 1 class(es), 62 instances
Done
```
That's all four classes of optimizations (P3/P15/P17/P18) engaging. Expected
decode: **~44 tok/s on M3 Ultra**, ~35-40 tok/s on M4 Max, ~25-30 tok/s on
M4 Pro.
## Minimum Hardware
| GPU | Min RAM | Notes |
|---|---|---|
| **M3 Ultra / M2 Ultra** | 96 GB | Tested on 256 GB, 44 tok/s |
| M4 Max | 96 GB | Expected ~35-40 tok/s |
| M4 Pro | 64 GB | Very tight; expect ~25-30 tok/s |
| M3 Max / M2 Max | 96 GB | Expected ~30-35 tok/s |
56.5 GB of GPU memory is needed just for the weights; add 2-5 GB for KV cache
and intermediate activations, plus enough system memory for the OS + other
processes.
## Why JANG for MiniMax
Standard MLX uniform quantization on MiniMax produces **completely broken
output at every bit level** — MMLU drops to ~25% (random guessing) because the
MoE router becomes unreliable. JANG's mixed-precision approach (attention +
router at full precision, routed experts at 2-bit) is the only working
quantized MiniMax on Apple Silicon.
JANGTQ takes this one step further by using a learned codebook for the 2-bit
expert weights. For MiniMax M2.5, JANG_2L (affine) scored 74% MMLU vs MLX's
25%. For MiniMax M2.7, **JANGTQ scores 91.5%** — the highest-quality
sub-60-GB MiniMax quant on any runtime.
---
## Compression Math
```
Quantization (offline, per weight matrix):
w_rot[r, i] = (H ⊙ signs * w^T)[r, i] # randomized Hadamard rotation
norms[r] = ||w_rot[r, :]||₂
packed[r, i] = argmin_c ||w_rot[r, i]/norms[r] - codebook[c]|| # Lloyd-Max 2-bit
Inference (runtime):
x_rot = H ⊙ (signs * x) # O(d log d) rotation
y[b, r] = norms[r] · Σᵢ x_rot[b, i] · codebook[unpack(packed[r, i])]
```
The Hadamard rotation flattens the heavy tail of the weight distribution, so a
4-entry codebook (2-bit) captures it with minimal error. The rotation is
symmetric (`H @ H = I`), so rotating the input once at runtime is
mathematically equivalent to rotating every weight once at quantization time.
Credit: [QuIP#](https://arxiv.org/abs/2402.04396) for the rotate-input-once
insight.
---
## Known Behaviors / Settings
- **Always-reasoning**: chat template opens `\n` at assistant start.
Give it `max_tokens ≥ 8192` in benchmarks.
- **Stop token**: single EOS `[e~[` = id 200020. `mlx_lm` reads this correctly
from `generation_config.json`.
- **Temperature 1.0 required**: greedy/temp=0 can cause the reasoning to get
stuck in a loop. Top-p 0.95 + top-k 40 recommended.
- **GPU RAM**: 56.5 GB base + KV cache grows with conversation length. Budget
60-65 GB for typical use, more for very long contexts.
---
**Created by Jinho Jang** (eric@jangq.ai) — part of the [JANG collection](https://huggingface.co/JANGQ-AI).
Base model: [MiniMaxAI/MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7).
Quantization method: JANGTQ (codebook + randomized Hadamard, see math above).
License: follows the upstream MiniMax open license.