# GLM-5.2-Int8Mix-NVFP4-REAP-594B

> Benchmarks in: **GPQA Diamond 86.87** (≈97% of full NVFP4) · **SciCode 47.77** (≈1.3 pts under full NVFP4). IFBench / τ²-Bench Telecom pending.

A REAP-pruned (≈22% of experts removed) Int8-mix NVFP4 quantization of GLM-5.2, ≈594B parameters.

## Evaluation

Measured under NVIDIA's evaluation protocol: **temperature=1.0, top_p=0.95**; **GPQA Diamond used max_new_tokens=100000**, others used max_new_tokens=64000 (SciCode via the official `inspect_ai` scorer, with-background). Full-model rows are NVIDIA's published figures for the **unpruned** GLM-5.2; the **REAP** rows are measured with [reap-bench](https://github.com/MadeBy561/reap-bench). **Intelligence lost** = relative drop vs full NVFP4 (same quant → isolates the prune itself).

| Model | GPQA Diamond | SciCode | IFBench | τ²-Bench Telecom |
|---|:-:|:-:|:-:|:-:|
| GLM-5.2 FP8 — full *(NVIDIA ref)* | 89.52 | 49.85 | 74.95 | 97.9 |
| GLM-5.2 NVFP4 — full *(NVIDIA ref)* | 89.39 | 49.04 | 75.81 | 98.25 |
| **GLM-5.2-Int8Mix-NVFP4-REAP-594B (this model)** · ~22% prune | **86.87** | **47.77** | — | — |
| ↳ *intelligence lost vs full NVFP4* | **−2.8%** | **−2.6%** | — | — |
| [GLM-5.2-NVFP4-REAP-504B-term](https://huggingface.co/madeby561/GLM-5.2-NVFP4-REAP-504B-term) · ~34% prune | — | 44.67 | — | — |
| ↳ *intelligence lost vs full NVFP4* | — | **−8.9%** | — | — |

GPQA Diamond: **172/198 correct, 0 errors** (reasoning_effort=max). SciCode (with-background): **139/291 subproblems = 47.77%**, **11/65 problems fully solved (16.92%)**, 65/65 samples, 0 errors. So far **≈97% of the full NVFP4 model's measured intelligence is retained** for an ≈22% expert prune — and on both axes the 594B clearly beats the more-aggressively-pruned [REAP-504B-term](https://huggingface.co/madeby561/GLM-5.2-NVFP4-REAP-504B-term) (168 experts). IFBench / τ²-Bench Telecom pending.

> Datasets: [GPQA Diamond](https://huggingface.co/datasets/Idavidrein/gpqa) (`gpqa_diamond.csv`, 198 Q) — Rein et al., [arXiv:2311.12022](https://arxiv.org/abs/2311.12022). [SciCode](https://github.com/scicode-bench/SciCode) via the official `inspect_ai` harness. Harness: [reap-bench](https://github.com/MadeBy561/reap-bench).