---
license: apache-2.0
base_model:
  - RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3
  - unsloth/gemma-4-26B-A4B-it-GGUF
base_model_relation: quantized
library_name: llama.cpp
tags:
  - gguf
  - llama.cpp
  - eagle3
  - speculative-decoding
  - speculator
  - draft-model
  - gemma-4
  - gemma
  - moe
  - redhatai
  - unsloth
pipeline_tag: text-generation

---

# Gemma 4 26B-A4B IT EAGLE3 Speculator GGUF

This repository contains GGUF conversions and quantizations of **RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3** for use with **llama.cpp EAGLE3 speculative decoding**.

> [!IMPORTANT]
> This is **not a standalone chat model**. It is an **EAGLE3 draft/speculator model** and must be used together with the matching target/verifier model.

* Target model: `unsloth/gemma-4-26B-A4B-it-GGUF`
* Speculator source: `RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3`
* Runtime: llama.cpp with `--spec-type draft-eagle3`

## Files

| File                                               | Type                             | Notes                                                         |
| -------------------------------------------------- | -------------------------------- | ------------------------------------------------------------- |
| `gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf`    | EAGLE3 speculator GGUF           | Converted from the original RedHatAI safetensors checkpoint   |
| `gemma-4-26B-A4B-it-speculator.eagle3-Q8_0.gguf`   | Quantized EAGLE3 speculator GGUF | Quantized from the F16 GGUF                                   |
| `gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf` | Quantized EAGLE3 speculator GGUF | Quantized from the F16 GGUF; may be faster for draft decoding |

## Usage with llama.cpp

Example:

```bash
llama-server \
  -m gemma-4-26B-A4B-it-Q4_K_M.gguf \
  -md gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf \
  --spec-type draft-eagle3 \
  --spec-draft-n-max 4 \
  --spec-draft-p-min 0.5 \
  -c 32768 \
  -ngl 99 \
  -fa on
```

Windows CMD example:

```cmd
llama-server.exe ^
  -m gemma-4-26B-A4B-it-Q4_K_M.gguf ^
  -md gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf ^
  --spec-type draft-eagle3 ^
  --spec-draft-n-max 4 ^
  --spec-draft-p-min 0.5 ^
  -c 32768 ^
  -ngl 99 ^
  -fa on
```

PowerShell example:

```powershell
.\llama-server.exe `
  -m "gemma-4-26B-A4B-it-Q4_K_M.gguf" `
  -md "gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf" `
  --spec-type draft-eagle3 `
  --spec-draft-n-max 4 `
  --spec-draft-p-min 0.5 `
  -c 32768 `
  -ngl 99 `
  -fa on
```

## Important Notes

This GGUF file is only the **draft/speculator model**. You still need a compatible GGUF of the target model, such as `unsloth/gemma-4-26B-A4B-it-GGUF`.

Do **not** use this speculator with unrelated models such as Gemma 4 12B, Gemma 4 31B, Gemma 3, Qwen, Llama, Mistral, or other non-matching models. EAGLE3 speculators are target-specific.

Even small differences in the target model, prompt format, quantization, or runtime settings may affect draft acceptance rate and overall speed.

## Tested Configuration

Tested with:

* Runtime: llama.cpp with EAGLE3 support
* Target model: `unsloth/gemma-4-26B-A4B-it-GGUF`
* Draft model: this EAGLE3 GGUF
* Example settings:

  * `--spec-type draft-eagle3`
  * `--spec-draft-n-max 4`
  * `--spec-draft-p-min 0.5`

Local benchmark observations may vary depending on GPU, quantization, context length, batch size, sampling settings, and prompt type.

## Benchmark Notes

In local testing, Gemma 4 26B-A4B IT without EAGLE3 already showed strong baseline decoding speed.

With EAGLE3 enabled, the draft acceptance rate was around `0.70` in local testing, with stronger gains on structured or predictable tasks such as:

* JSON output
* stepwise math
* code completion
* summarization
* long reasoning
* repeated pattern generation

It was less effective on some open-ended or language-sensitive tasks such as:

* translation
* creative writing
* general explanation
* some factual QA prompts

On this model, EAGLE3 may be useful for structured output, agent/tool-style responses, code completion, and predictable formats. For general chat, translation, roleplay, or creative writing, the non-speculative baseline may be competitive or more consistent.

On smaller VRAM setups, the extra draft/speculator model may reduce the practical benefit of EAGLE3. In those cases, native MTP models or the base Gemma 4 26B-A4B model without speculative decoding may be more efficient.

## Conversion

Converted with llama.cpp `convert_hf_to_gguf.py` using the original speculator repository and the matching target model directory.

Example conversion command:

```bash
python convert_hf_to_gguf.py \
  RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3 \
  --outtype f16 \
  --target-model-dir gemma-4-26B-A4B-it \
  --outfile gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf
```

PowerShell example:

```powershell
python .\convert_hf_to_gguf.py `
  "E:\OLLAMA_MODELS\gemma-4-26B-A4B-it-speculator.eagle3" `
  --outtype f16 `
  --target-model-dir "E:\OLLAMA_MODELS\gemma-4-26B-A4B-it" `
  --outfile "E:\OLLAMA_MODELS\gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf"
```

## Quantization

The F16 GGUF can be quantized with `llama-quantize`.

Q8_0 example:

```bash
llama-quantize \
  gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf \
  gemma-4-26B-A4B-it-speculator.eagle3-Q8_0.gguf \
  Q8_0
```

Q4_K_M example:

```bash
llama-quantize \
  gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf \
  gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf \
  Q4_K_M
```

PowerShell example:

```powershell
.\llama-quantize.exe `
  "E:\OLLAMA_MODELS\gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf" `
  "E:\OLLAMA_MODELS\gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf" `
  Q4_K_M
```

## Credits

Original EAGLE3 speculator model by RedHatAI:

* `RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3`

Target GGUF model:

* `unsloth/gemma-4-26B-A4B-it-GGUF`

GGUF support and runtime:

* `ggml-org/llama.cpp`

## License

This repository is a converted GGUF version of the original speculator model. The original model license and usage terms apply. Please refer to the upstream repositories for full license details.