---
library_name: mlx
license: apache-2.0
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
pipeline_tag: any-to-any
base_model: google/gemma-4-12B-it-qat-q4_0-unquantized
tags:
- mlx
---
This model is quantized from `google/gemma-4-12B-it-qat-q4_0-unquantized` using the command `python -m mlx_vlm convert --hf-path google/gemma-4-12B-it-qat-q4_0-unquantized -q --q-group-size 32 --q-bits 4 --mlx-path "./gemma-4-12B-it-qat-4bit-gs32"`. It uses the correct group size of 32, which corresponds to the 32 nibbles (uint4) of [Q4_0](https://github.com/ggml-org/whisper.cpp/blob/a8ec021f2750a473ff4a8f3883bc9fdf5feafa84/ggml/src/ggml-opencl/kernels/mul_mv_q4_0_f32.cl#L21).

Simply put, the default of mlx-lm and mlx-vlm is 64 group size, which does not match the QAT parameters that Google is using, since that targets llama.cpp, which uses 32 by default. This quantization fixes that at the cost of slightly inflated quant size (scale parameter storage size is effectively doubled).