---
language: en
library_name: mlx
pipeline_tag: image-text-to-text
tags:
- mlx
---

# eyes-ml/gemma-4-12B-it-qat-mlx-4bit

## note: quantized using a [patched version of mlx_vlm.convert](https://github.com/sockeye44/mlx-vlm/blob/feat/convert-cli-ignore-args/mlx_vlm/convert.py):

```bash
mlx_vlm.convert --hf-path eyes-ml/gemma-4-12B-it-qat4_0-bf16 --mlx-path ./gemma-4-12B-it-qat-mlx-4bit -q --q-bits 4 --ignore-quant-preset gemma4 --upload-repo eyes-ml/gemma-4-12B-it-qat-mlx-4bit
```

## Use with mlx

```bash
pip install -U mlx-vlm
```

```bash
python -m mlx_vlm.generate --model eyes-ml/gemma-4-12B-it-qat-mlx-4bit --max-tokens 100 --temperature 0.0 --prompt "Describe this image." --image <path_to_image>
```