---
license: mit
base_model:
- google/gemma-4-12B-it-qat-q4_0-unquantized-assistant
tags:
- gguf
- qat
- mtp
- gemma
---

# Gemma 4 12B QAT MTP Assistant GGUF

GGUF quantizations converted from `google/gemma-4-12B-it-qat-q4_0-unquantized-assistant`.

Tested with llama.cpp b9549+ (Gemma 4 MTP support).

## Recommendations

- `Q4_0_Q4emb` — recommended for most users
- `Q8_0` — highest precision

## Files

* `gemma-4-12b-qat-it-assistant-Q4_0-Q6emb.gguf`
* `gemma-4-12b-qat-it-assistant-Q4_0_Q4emb.gguf`
* `gemma-4-12b-qat-it-assistant-Q8_0.gguf`

### Notes

`Q4_0_Q4emb` keeps `token_embd.weight` in `Q4_0` rather than the higher-precision embedding quantization normally used by llama.cpp.

Initial testing showed similar draft acceptance rates while reducing file size.

## Example

```bash
llama-server \
  -m gemma-4-12B-it-qat-Q4_0.gguf \
  -md gemma-4-12b-qat-it-assistant-Q4_0_Q4emb.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 2