---
base_model: zai-org/GLM-5.2
base_model_relation: quantized
license: mit
tags:
- glm_moe_dsa
- conversational
- ik_llama.cpp
---

# GLM-5.2-GGUF-2.788bpw

This is a 2.8 BPW quantized model for the GPU riches with more combined RAM + VRAM than common sense.

The quant aims to achieve best-in-class performance, by relying on SOTA quants from ik_llama.cpp:
* Routed experts tensors use the **IQ2_KL** quant (2.6875 BPW)
* Indexer tensors use the **Q8_0** quant (8.5 BPW)
* All other tensors use the **Q6_0** quant (6.5 BPW)

Coupled with the recent enhancements:
* MTP support with e.g. `--spec-type mtp:n_max=4,p_min=0.0` ([#1890](https://github.com/ikawrakow/ik_llama.cpp/pull/1890))
* graph parallel support with `-sm graph` ([#1821](https://github.com/ikawrakow/ik_llama.cpp/pull/1821))
* DSA support with `-dsa -fidx` ([#2045](https://github.com/ikawrakow/ik_llama.cpp/pull/2045), [#2098](https://github.com/ikawrakow/ik_llama.cpp/pull/2098), [#2109](https://github.com/ikawrakow/ik_llama.cpp/pull/2109), and many others)

it should run at decent speed as well, with very little slowdown at long context.

(Note: For now, `-sm graph` and quantize KV cache e.g. `-ctk q8_0` do not work together with `-dsa -fidx`. As a fun exercise, you can ask this quant to get them to work.)


## Versions

There are 2 versions:
* [GLM-5.2-GGUF-2.788bpw.gguf](GLM-5.2-GGUF-2.788bpw.gguf) - Made with the [imatrix from unsloth](https://huggingface.co/unsloth/GLM-5.2-GGUF/blob/main/imatrix_unsloth.gguf_file) (thanks!)
* [GLM-5.2-GGUF-2.788bpw-muzzy-imatrix.gguf](GLM-5.2-GGUF-2.788bpw-muzzy-imatrix.gguf) - Made with the [imatrix from muzzy](https://huggingface.co/muzzy/GLM-5.2-GGUF/blob/main/GLM5.2-imatrix-muzzy.gguf) (thanks!)

Comparison:
| version                                  | imatrix | indexer | ppl                |
| ---------------------------------------- | ------- | ------- | ------------------ |
| GLM-5.2-GGUF-2.788bpw.gguf               | unsloth | Q8_0    | 3.2731 +/- 0.01793 |
| GLM-5.2-GGUF-2.788bpw-muzzy-imatrix.gguf | muzzy   | Q8_0    | 3.2172 +/- 0.01751 |

The PPL results were gathered using llama-perplexity with `-f wiki.test.raw -c 512`.

**Recommendation:** GLM-5.2-GGUF-2.788bpw-muzzy-imatrix.gguf - The imatrix from muzzy, which took him 5 days to generate, is just amazing.


## Size

Size from `llama-server` output:
```
llm_load_print_meta: model size       = 244.483 GiB (2.788 BPW)
llm_load_print_meta: repeating layers = 243.043 GiB (2.778 BPW, 751.427 B parameters)
```

Buffer size with `-cmoe --no-mmap`:
```
llm_load_tensors:        CPU buffer size = 233312.34 MiB
llm_load_tensors:      CUDA0 buffer size = 13724.12 MiB
```

<details>

<summary>Recipe</summary>

```
# Attention
blk\..*\.attn_k_b\.weight=q6_0
blk\..*\.attn_v_b\.weight=q6_0

blk\..*\.attn_kv_a_mqa\.weight=q6_0
blk\..*\.attn_q_a\.weight=q6_0
blk\..*\.attn_q_b\.weight=q6_0
blk\..*\.attn_output\.weight=q6_0

# First 3 Dense Layers
blk\..*\.ffn_down\.weight=q6_0
blk\..*\.ffn_(gate|up)\.weight=q6_0

# Shared Expert Layers
blk\..*\.ffn_down_shexp\.weight=q6_0
blk\..*\.ffn_(gate|up)_shexp\.weight=q6_0

# Routed Experts Layers
blk\..*\.ffn_(up|gate|down)_exps\.weight=iq2_kl

# Indexer
blk\..*\.indexer\.proj\.weight=q8_0
blk\..*\.indexer\.attn_k\.weight=q8_0
blk\..*\.indexer\.attn_q_b\.weight=q8_0

# NextN MTP Layer
blk\..*\.nextn\.embed_tokens\.weight=q6_0
blk\..*\.nextn\.shared_head_head\.weight=q6_0
blk\..*\.nextn\.eh_proj\.weight=q6_0

# Non-Repeating Layers
token_embd\.weight=q6_0
output\.weight=q6_0
```
</details>


## Flags

Example that uses DSA with IndexShare:
```
--no-mmap -ngl 99 --fit \
-mla 1 -dsa -fidx \
-c 102400 \
-b 2048 -ub 2048 -wgt 1 \
-cram 0 -muge -cuda graphs=1 \
--jinja --parallel-tool-calls \
--chat-template-kwargs {"reasoning_effort": "high"} \
--spec-type mtp:n_max=4,p_min=0.5
```


## Comparison with other quants

| quant                                    | size                        | ppl                    |
| ---------------------------------------- | --------------------------- | ---------------------- |
| GLM-5.2-GGUF-2.788bpw-muzzy-imatrix.gguf | **244.483 GiB (2.788 BPW)** | 3.2172 +/- 0.01751     |
| IQ2_KL from muzzy/GLM-5.2-GGUF           | 261.988 GiB (2.985 BPW)     | **3.1085 +/- 0.01690** |
| UD-IQ3_XXS from unsloth/GLM-5.2-GGUF     | 262.334 GiB (2.989 BPW)     | 3.2952 +/- 0.01824     |

The IQ2_KL from https://huggingface.co/muzzy/GLM-5.2-GGUF, using the same recipe as the IQ2_KL from https://huggingface.co/ubergarm/GLM-5.1-GGUF, is about 17.5 GiB larger than this, and has a better PPL. Recommended if there is enough RAM + VRAM.

On the other hand, the UD-IQ3_XXS from https://huggingface.co/unsloth/GLM-5.2-GGUF is even larger, while having a worse PPL. Unless one is stuck with the mainline llama.cpp, there is no good reason to use this unsloth quants.