---
license: apache-2.0
tags:
- llama-cpp
- gguf
- quantized
base_model: google/gemma-4-12B-it
pipeline_tag: image-text-to-text
---

# gemma-4-12B-it-GGUF

Quantized GGUF versions of [google/gemma-4-12B-it](https://huggingface.co/google/gemma-4-12B-it),
created with [llama.cpp](https://github.com/ggml-org/llama.cpp).


## Usage

### With llama.cpp
```bash
llama-cli -hf MoMonir/gemma-4-12B-it-GGUF
```

### With llama-cpp-python
```python
from llama_cpp import Llama
llm = Llama.from_pretrained(repo_id="MoMonir/gemma-4-12B-it-GGUF", filename="google_gemma-4-12B-it-Q8_0.gguf")
response = llm("Hello, world!", max_tokens=100)
print(response["choices"][0]["text"])
```

## License
Please refer to the [original model license](https://huggingface.co/google/gemma-4-12B-it).