---
license: gemma
base_model: google/gemma-4-E4B-it-assistant
tags:
  - gguf
  - llama.cpp
  - gemma4
  - mtp
  - speculative-decoding
library_name: gguf
---

# Gemma 4 E4B — MTP (assistant) head · GGUF

GGUF conversions of **`google/gemma-4-E4B-it-assistant`**, the Gemma 4
Multi-Token-Prediction (MTP) *assistant head* — architecture `gemma4-assistant`
in llama.cpp.

> **These are not a standalone chat model.** This is the small speculative /
> *draft-mtp* head (~76M parameters) that runs **alongside** a Gemma 4 E4B main
> model to accelerate generation via speculative decoding. Load it as the draft
> model, not as the primary model.

## Files

| File | Type |
|------|------|
| `gemma-4-E4B-it-assistant-f16.gguf` | F16 |
| `gemma-4-E4B-it-assistant-Q4_0.gguf` | Q4_0 |
| `gemma-4-E4B-it-assistant-Q6_K.gguf` | Q6_K |
| `gemma-4-E4B-it-assistant-Q8_0.gguf` | Q8_0 |

All variants are converted from the original bf16 checkpoint with
[llama.cpp](https://github.com/ggml-org/llama.cpp) (commit `f955e39`).

## Choosing a quantization

Most of this head's weights live in the token-embedding tensor, so quantization
mainly shrinks the embedding:

- **F16** — highest draft **acceptance rate**; recommended when disk isn't a concern.
- **Q8_0** — near-F16 quality, ~half the size.
- **Q6_K** — smaller still, minor acceptance loss.
- **Q4_0** — smallest; acceptance drops the most (the embedding is quantized to 4-bit).

Because the head is tiny to begin with, the size savings from lower bit-widths
are modest while acceptance decreases — **F16 or Q8_0 are the practical picks**.

## Usage (llama.cpp speculative decoding)

Use this file as the draft/MTP head together with a Gemma 4 E4B main model in a
llama.cpp build that supports `gemma4` MTP (`draft-mtp`) speculative decoding.
The head shares the KV cache with the target model. Typical speculative-decoding
settings: `n_max` (draft length) of **3** tends to give the best throughput;
higher values reduce acceptance and can end up slower than no speculation.

## Base model & license

Derived from the gated model
[`google/gemma-4-E4B-it-assistant`](https://huggingface.co/google/gemma-4-E4B-it-assistant).
Distributed under the [Gemma license](https://ai.google.dev/gemma/terms);
you are responsible for complying with Google's Gemma Terms of Use.