eyes-ml/gemma-4-12B-it-qat-mlx-4bit

note: quantized using a patched version of mlx_vlm.convert:

mlx_vlm.convert --hf-path eyes-ml/gemma-4-12B-it-qat4_0-bf16 --mlx-path ./gemma-4-12B-it-qat-mlx-4bit -q --q-bits 4 --ignore-quant-preset gemma4 --upload-repo eyes-ml/gemma-4-12B-it-qat-mlx-4bit

Use with mlx

pip install -U mlx-vlm

python -m mlx_vlm.generate --model eyes-ml/gemma-4-12B-it-qat-mlx-4bit --max-tokens 100 --temperature 0.0 --prompt "Describe this image." --image <path_to_image>