Hugging Face's logo

kaitchup
/
GLM-4.7-Flash-autoround-w4a16-excl-sharedexperts

Safetensors
glm4_moe_lite
autoround
4-bit precision
auto-round
Model card Files
xet
 Community
GLM-4.7-Flash-autoround-w4a16-excl-sharedexperts / README.md
bnjmnmarie's picture
bnjmnmarie
Create README.md
5a3282e verified
preview code
|
raw
history blame contribute delete
968 Bytes
metadata 
base_model:
  - zai-org/GLM-4.7-Flash
tags:
  - autoround
license: mit

This is zai-org/GLM-4.7-Flash quantized with AutoRound to W4A16, with fallback to 16-bit for the first MLP layer and the shared experts. The model is compatible with vLLM (tested: v0.15.0). Tested with an RTX Pro 6000 WK.