---
license: mit
language:
- en
- zh
library_name: mlx
pipeline_tag: text-generation
tags:
- mlx
- apple-silicon
- jang
- glm
- glm5
- moe
- mixture-of-experts
- mla
- dsa
- quantized
- 2bit
- experimental
- research-artifact
base_model: zai-org/GLM-5.2
base_model_relation: quantized
inference: false
extra_gated_heading: "Access is manual — please read the warning first"
extra_gated_prompt: >-
  **There is no working runtime for this bundle.** vMLX, Osaurus, MLX Studio and
  stock `mlx_lm` cannot load it today. It is a 318 GB research artifact published
  for archival and for researchers working on SSD-offloaded MoE inference. It
  will not fit in RAM on any shipping Apple Silicon machine, and there is no
  supported way to generate text with it. Requests are reviewed manually — please
  say what you intend to do with it.
extra_gated_fields:
  Name: text
  Affiliation: text
  What do you intend to use this bundle for?: text
  I understand there is no working runtime and this will not run out of the box: checkbox
extra_gated_button_content: "Request access"
---

> # ⚠️ NO RUNTIME AVAILABLE YET — DO NOT EXPECT THIS TO RUN
>
> **There is no vMLX / Osaurus / MLX Studio runtime for this bundle.** Stock `mlx_lm`
> will not load it either.
>
> These weights are published as an **archival research artifact**, not as a usable
> model. At 318 GB they do not fit in RAM on any shipping Apple Silicon machine, and
> the only code that has ever produced coherent text from them is an unreleased
> SSD-offload prototype (`mlxpress`) that tops out at **~2.8 tok/s** and was shelved
> for missing its performance bar.
>
> **If you want a GLM MoE bundle you can actually run today, use
> [`GLM-5.1-JANG_1L`](https://huggingface.co/JANGQ-AI/GLM-5.1-JANG_1L) instead.**

---

# GLM-5.2-JANG_2L

**745B-parameter Mixture-of-Experts quantized to 3.12 bits/weight — 318 GB**
**Created by Jinho Jang — eric@jangq.ai**

A JANG v2 affine mixed-precision quantization of [`zai-org/GLM-5.2`](https://huggingface.co/zai-org/GLM-5.2),
produced 2026-07-19 with `jang_tools convert -p JANG_2L`. The conversion is complete and
structurally verified; **what does not exist is a runtime that can serve it.**

---

## Why this is here

This bundle was built for the `mlxpress` campaign — an attempt to run a 745B MoE on a
128 GB Apple Silicon machine by keeping only hot experts resident and streaming cold
experts from SSD. The quantization worked. The offload engine worked and produced
coherent long-context output. It was **too slow to ship** (~2.8 tok/s against a 6 tok/s
bar), so the campaign was paused and no runtime was ever released.

Rather than delete a 318 GB artifact that took a 1.5 TB download and a multi-hour
conversion to produce, it is archived here. It is useful if you are working on
SSD-offloaded MoE inference, expert-routing analysis, or low-bit quantization of
DeepSeek-V3.2-family architectures.

---

## Model summary

| Field | Value |
|---|---|
| Base model | [`zai-org/GLM-5.2`](https://huggingface.co/zai-org/GLM-5.2) (bf16 source, not FP8) |
| Architecture | `GlmMoeDsaForCausalLM` / `model_type=glm_moe_dsa` (DeepSeek-V3.2 family + DSA sparse attention) |
| Parameters | ~745 B total, ~40 B active per token |
| Layers | 78 transformer layers + layer 78 = MTP (**MTP is present in this bundle, not stripped**) |
| Routed experts | 256, top-8, sigmoid routing (`noaux_tc`), `routed_scaling_factor=2.5` |
| Shared experts | 1 |
| Dense layers | first 3 (`first_k_dense_replace=3`), `intermediate_size=12288` |
| Expert MLP | `moe_intermediate_size=2048` |
| Hidden size | 6144 |
| Attention | MLA — `q_lora_rank=2048`, `kv_lora_rank=512`, 64 heads, `qk_head_dim=256`, `v_head_dim=256` |
| DSA indexer | 32 heads, `index_head_dim=128`, `index_topk=2048`, `index_topk_freq=4` |
| RoPE | `theta=8e6`, interleaved |
| Context window | 1,048,576 (1M) per config |
| Vocab | 154,880 |
| EOS tokens | `[154820, 154827, 154829]` |
| Format | JANG v2 — MLX-native safetensors, mmap load |
| On-disk size | **318 GiB** (341,207,533,424 bytes) across 233 shards, 3,303 tensors |
| Mode | Text-only |

---

## Quantization

Profile **`JANG_2L`**, affine, via `mx.quantize`. No AWQ, no Hadamard rotation, no
expert pruning, no TurboQuant/JANGTQ anywhere in this bundle.

| Setting | Value |
|---|---|
| Method | `jang-importance` |
| Target bits | 2.0 |
| **Actual bits** | **3.12** |
| Group size | 64 |
| Scheme | asymmetric affine |
| Backend | `mx.quantize` |
| Quantization objective | MSE |
| Scoring | weight-magnitude |
| Calibration | weights (no activation calibration) |
| MLP asymmetry floor | enabled |
| Hadamard rotation | disabled |
| Bit widths used | 2, 3, 4, 6, 8 |
| Passthrough (unquantized) tensors | 0 |
| Manifest | 929 entries, schema v1, in `jang_config.json` |

Notable per-tensor assignments:

| Tensor class | Bits |
|---|---|
| `lm_head` | 8 |
| `embed_tokens` | 6 |
| dense-layer MLP (layers 0–2) | 8 |
| routed experts (`switch_mlp.{gate,up,down}_proj`) | 2 (bulk of the model) |

The 3.12 actual-bits figure against a 2.0 target is why this landed at 318 GB rather
than the ~300 GB the campaign was aiming for — the asymmetry floor and the 8-bit
high-precision slice cost more than budgeted.

---

## Measured performance

All numbers below are from the unreleased `mlxpress` Python offload prototype on an
**M5 Max MacBook, 128 GB**, experts streamed from internal NVMe, 45 GB hot cache,
sampled (temp 0.7 / top-p 0.95 / rep-pen 1.1). These are the only numbers that exist
for this bundle.

| Routing config | Experts kept / 8 | tok/s | Coherent |
|---|---|---|---|
| exact (all 8) | 8.0 | 0.34–0.40 | ✅ |
| min-keep 6 by router weight | 6.2 | 0.67 | ✅ |
| min-keep 5 by router weight | 5.4 | 1.25 | ✅ |
| **min-keep 4 by router weight** | **4.6** | **1.85** | ✅ |
| min-keep 4 + multithreaded reads | 4.6 | **~2.8 (best stable)** | ✅ |
| pure force-resident (drop by residency) | 2.5 | 6.9 | ❌ garbage |

**No standard benchmarks (MMLU, HumanEval, GSM8K) were run on this bundle.** The
campaign was paused before evaluation. Quality was assessed only by manual inspection
of generated text, which was coherent at every ✅ row above.

Two findings from those runs that may be useful to others:

- **Coherence needs only the ~4–5 highest-router-weight experts per layer**, not all 8.
  Dropping the low-weight tail stays coherent down to 4.6/8. Dropping by *cache
  residency* instead of by router weight discards high-weight experts and immediately
  produces garbage.
- **Shrink-to-fit is not available for this model.** Expert weight matrices were
  measured to be effectively full-rank and mutually orthogonal, so there is no
  low-rank or shared-basis trick that makes 40 B active parameters fit the ~50 GB of
  spendable cache on a 128 GB machine.

---

## Known issues

1. **`lm_head` is mislabeled.** The three `lm_head` tensors are stored under the
   `language_model.` prefix (`language_model.lm_head.{weight,scales,biases}`) while
   all 3,300 other tensors use `model.`. This is a converter bug. Any loader must
   remap it — the offload prototype had to dequantize it and assign it as a plain
   fp16 tensor to get coherent output. **Loading this bundle without handling that
   produces pure garbage**, which is exactly how the bug was found.
2. **MTP layer 78 is included.** `model.layers.78.eh_proj` and its own MoE block are
   present. Any loader must either use or explicitly strip them. MTP was evaluated for
   the offload path and ruled out: verifying K speculative tokens touches K× distinct
   experts, so drive-bound cost scales with distinct experts read, not with forward
   passes.
3. **318 GB does not fit in RAM** on any shipping Apple Silicon machine. SSD offload
   is mandatory, and no released runtime implements it.
4. **Not evaluated.** See the benchmarks note above.

---

## Files

| File | Purpose |
|---|---|
| `model-*-of-00233.safetensors` | 233 weight shards |
| `model.safetensors.index.json` | shard map, 3,303 tensors |
| `config.json` | architecture + MLX `quantization` block (per-tensor overrides) |
| `jang_config.json` | full JANG quantization manifest (929 entries) |
| `generation_config.json` | sampling + stop tokens |
| `tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja` | tokenizer |

---

## License

MIT, inherited from [`zai-org/GLM-5.2`](https://huggingface.co/zai-org/GLM-5.2).

## Credit

Quantization, offload research, and this bundle by **Jinho Jang** — eric@jangq.ai.
Base model by Z.ai (Zhipu AI).