---
license: mit
base_model:
- zai-org/GLM-5.2
---
# Model Overview

- **Model Architecture:** GLM-5.2
  - **Input:** Text
  - **Output:** Text
- **Supported Hardware Microarchitecture:** AMD MI350/MI355
- **ROCm:** 7.0.0
- **PyTorch:** 2.9.0
- **Transformers:** 5.13.1
- **Operating System(s):** Linux
- **Inference Engine:** [vLLM](https://docs.vllm.ai/en/latest/)
- **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) (V0.12)
  - **Weight quantization:** OCP MXFP4, Static; self_attn Per-channel, FP8E4M3, Static
  - **Activation quantization:** OCP MXFP4, Dynamic; self_attn Per-token, FP8E4M3, Dynamic

This model was built with GLM-5.2 model by applying [AMD-Quark](https://quark.docs.amd.com/latest/index.html) for MXFP4 and PTPC FP8 quantization.

# Model Quantization

The model was quantized from [zai-org/GLM-5.2](https://huggingface.co/zai-org/GLM-5.2) using [AMD-Quark](https://quark.docs.amd.com/latest/index.html). 

**Quantization scripts:**

```bash
cd Quark/examples/torch/language_modeling/llm_ptq/
python quantize_quark.py \
  --model_dir zai-org/GLM-5.2 \
  --output_dir "$OUTPUT_DIR" \
  --quant_scheme mxfp4 \
  --layer_quant_scheme "*self_attn*" ptpc_fp8 \
  --exclude_layers \
      "*lm_head" \
      "*embed_tokens" \
      "*eh_proj" \
      "*mlp.gate" \
      "*self_attn.indexer.weights_proj" \
  --file2file_quantization
```

# Deployment
### Use with vLLM

This model can be deployed efficiently using [vLLM](https://docs.vllm.ai/en/latest/).

## Evaluation
The model was evaluated on GSM8K and GPQA-Diamond-CoT benchmarks. 

### Accuracy

<table>
  <tr>
    <td><strong>Benchmark</strong>
    </td>
    <td><strong>zai-org/GLM-5.2</strong>
    </td>
    <td><strong>amd/GLM-5.2-Quark-MXFP4-AttnFP8 (this model)</strong>
    </td>
    <td><strong>Recovery</strong>
    </td>
  </tr>
  <tr>
    <td>GSM8K (flexible-extract)
    </td>
    <td>96.46
    </td>
    <td>95.80
    </td>
    <td>99.3%
    </td>
  </tr>
  <tr>
    <td>GPQA-Diamond-CoT (flexible-extract)
    </td>
    <td>69.53
    </td>
    <td>68.35
    </td>
    <td>98.3%
    </td>
  </tr>
</table>

### Reproduction

The above evaluation results were obtained using the `lm-evaluation-harness` framework, based on the Docker image `rocm/vllm-dev:nightly_main_20260712`, with vLLM and lm-eval compiled and installed from source.

```
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_USE_AITER_FP8BMM=0
export VLLM_ROCM_USE_AITER_FP4BMM=0

lm_eval \
        --model vllm \
        --model_args "pretrained=amd/GLM-5.2-Quark-MXFP4-AttnFP8,tensor_parallel_size=4,dtype=auto,quantization=quark,gpu_memory_utilization=0.9,max_model_len=40960,trust_remote_code=True" \
        --tasks gsm8k gpqa_diamond_cot_zeroshot \
        --apply_chat_template \
        --gen_kwargs "max_gen_toks=32768,temperature=1.0,top_p=0.95,do_sample=True" \
        --batch_size auto
```

# License
Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.