---
library_name: gguf
license: mit
base_model: zai-org/GLM-5.2
base_model_relation: quantized
language:
  - en
  - zh
  - es
  - de
  - pt
  - it
  - fr
tags:
  - gguf
  - glm-dsa
  - moe
  - imatrix
  - iq2_s
  - iq4_nl
  - mixed-precision
  - llama-cpp
quantized_by: Deviad
---

# GLM-5.2 — Mixed-Precision GGUF (IQ2_S experts · IQ4_NL rest)

Mixed-bit GGUF re-quantization of **zai-org/GLM-5.2** (256×22B Mixture-of-Experts,
architecture `glm-dsa`), produced with `llama-quantize`'s per-tensor mixed-precision
workflow and an importance matrix.

The target of this quant is **memory footprint**: MoE expert fragments are dropped
to **IQ2_S** (≈2.64 BPW overall) while the dense/sensitive tensors (attention, norms,
shared components, embeddings) stay at **IQ4_NL**. The last MoE block (`blk.78.*`) is
kept at IQ4_NL because `llama-quantize` does not find separate weights for it (it is
covered by the shared/head tensors) — see the mapping table below.

## Model particulars (from GGUF KV metadata)

| Key | Value |
|---|---|
| Architecture | `glm-dsa` |
| Name / version | GLM-5.2 / 5.2 |
| Size label | 256x22B (256 experts, 8 active, 1 shared) |
| Block count | 79 (3 leading dense + 76 MoE) |
| Context length | 1,048,576 (1M tokens) |
| Embedding length | 6144 |
| Feed-forward length (dense) | 12288 |
| Expert FF length | 2048 |
| Attention heads / KV heads | 64 / 1 (MLA, `q_lora_rank=2048`, `kv_lora_rank=512`, `key_length_mla=256`, `value_length_mla=256`) |
| RoPE base / dim | 8,000,000 / 64 |
| Vocabulary | 154,880 (tokenizer `glm4` / `gpt2`) |
| Expert gating | func=2, weights_scale=2.5, weights_norm=true |
| NextN predict layers | 1 |
| License | MIT |

## Quantization mapping

Per-tensor type assignment passed to `llama_quantize` (`--tensor-type` overrides):

| Tensor pattern | Quant |
|---|---|
| `blk.78.ffn_down_exps` | IQ4_NL |
| `blk.78.ffn_gate_exps` | IQ4_NL |
| `blk.78.ffn_up_exps` | IQ4_NL |
| `ffn_gate_exps` (all other blocks) | IQ2_S |
| `ffn_up_exps` (all other blocks) | IQ2_S |
| `ffn_down_exps` (all other blocks) | IQ2_S |
| everything else (attention, norms, embeddings, shared head, indexer) | IQ4_NL |

- **Source GGUF**: `unsloth/GLM-5.2-GGUF` → this repo's `UD-IQ4_NL` variant (9 shards, IQ4_NL),
  re-quantized here with `allow-requantize` + `keep-split`.
- **Importance matrix**: `imatrix_unsloth.gguf` (sourced from Unsloth).
- **Final size**: ≈232 GiB across 9 shards, **≈2.64 BPW** (input ≈3.95 BPW at q8_0).

## Files

Filenames include the `IQ2_S`/`IQ4_NL` quant tokens so Hugging Face's
quantization-variant scanner recognizes the shards (single-quant names are not
possible for a mixed-precision quant; both constituent quants are listed).

| File | Size |
|---|---|
| `GLM-5.2-mixed-IQ2_S-IQ4_NL-00001-of-00009.gguf` | 9.0 MiB (headers/tokenizer) |
| `GLM-5.2-mixed-IQ2_S-IQ4_NL-00002-of-00009.gguf` | 29.9 GiB |
| `GLM-5.2-mixed-IQ2_S-IQ4_NL-00003-of-00009.gguf` | 31.0 GiB |
| `GLM-5.2-mixed-IQ2_S-IQ4_NL-00004-of-00009.gguf` | 31.1 GiB |
| `GLM-5.2-mixed-IQ2_S-IQ4_NL-00005-of-00009.gguf` | 31.0 GiB |
| `GLM-5.2-mixed-IQ2_S-IQ4_NL-00006-of-00009.gguf` | 31.0 GiB |
| `GLM-5.2-mixed-IQ2_S-IQ4_NL-00007-of-00009.gguf` | 31.1 GiB |
| `GLM-5.2-mixed-IQ2_S-IQ4_NL-00008-of-00009.gguf` | 31.0 GiB |
| `GLM-5.2-mixed-IQ2_S-IQ4_NL-00009-of-00009.gguf` | 16.2 GiB |

## Usage

Load with any recent `llama.cpp` build (and compatible runners — LM Studio, Ollama,
koboldcpp, etc.) that supports the `glm-dsa` architecture, MLA attention and IQ2_S /
IQ4_NL dequantization (GPU offload strongly recommended).

```bash
llama-server \
  -m GLM-5.2-mixed-IQ2_S-IQ4_NL-00001-of-00009.gguf \
  --host 0.0.0.0 --port 8080 \
  -ngl 999 -c 8192
```

> The first shard is treated as the entry point; `llama.cpp` follows the split-file
> links to load all 9 shards automatically. Point `-m` at `00001-of-00009`.

## Provenance

- **Base model**: [zai-org/GLM-5.2](https://huggingface.co/zai-org/GLM-5.2) — MIT.
- **Source GGUF quantization**: Unsloth (`general.quantized_by = Unsloth`,
  `general.repo_url = https://huggingface.co/unsloth`).
- **This mixed-precision re-quant + imatrix**: Deviad (2026-06-20),
  on Apple M3 Ultra (Metal 4 build of `llama.cpp`, build 1 / `4b48a53`).

## Disclaimer

This is an aggressive low-bit quantization intended to fit a very large MoE into
constrained memory. Expect measurable quality degradation versus the source,
especially because the expert tensors are at IQ2_S. Validate on your own tasks.