---
license: apache-2.0
base_model:
  - RedHatAI/Qwen3-8B-speculator.eagle3
  - Qwen/Qwen3-8B
base_model_relation: quantized
library_name: llama.cpp
tags:
  - gguf
  - llama.cpp
  - eagle3
  - speculative-decoding
  - speculator
  - draft-model
  - qwen3
  - qwen
  - redhatai
pipeline_tag: text-generation
---

# Qwen3-8B EAGLE3 Speculator GGUF

This repository contains a GGUF conversion of **RedHatAI/Qwen3-8B-speculator.eagle3** for use with **llama.cpp EAGLE3 speculative decoding**.

> [!IMPORTANT]
> This is **not a standalone chat model**. It is an **EAGLE3 draft/speculator model** and must be used together with the matching target/verifier model.

- Target model: `Qwen/Qwen3-8B`
- Speculator source: `RedHatAI/Qwen3-8B-speculator.eagle3`
- Runtime: llama.cpp with `--spec-type draft-eagle3`

## Files

| File | Type | Notes |
|---|---|---|
| `Qwen3-8B-speculator.eagle3-F16.gguf` | EAGLE3 speculator GGUF | Converted from the original RedHatAI safetensors checkpoint |

## Usage with llama.cpp

Example:

```bash
llama-server \
  -m Qwen3-8B-Q4_K_M.gguf \
  -md Qwen3-8B-speculator.eagle3-F16.gguf \
  --spec-type draft-eagle3 \
  --spec-draft-n-max 4 \
  --spec-draft-p-min 0.5 \
  -c 32768 \
  -ngl 99 \
  -fa on
```

Windows CMD example:

```cmd
llama-server.exe ^
  -m Qwen3-8B-Q4_K_M.gguf ^
  -md Qwen3-8B-speculator.eagle3-F16.gguf ^
  --spec-type draft-eagle3 ^
  --spec-draft-n-max 4 ^
  --spec-draft-p-min 0.5 ^
  -c 32768 ^
  -ngl 99 ^
  -fa on
```

## Important Notes

This GGUF file is only the **draft/speculator model**. You still need a compatible GGUF of the target model, such as a GGUF conversion or quantization of `Qwen/Qwen3-8B`.

Do **not** use this speculator with unrelated models such as Qwen3-14B, Qwen3-32B, Gemma, Llama, or Qwen3.6 models. EAGLE3 speculators are target-specific.

## Tested Configuration

Tested with:

- Runtime: llama.cpp with EAGLE3 support
- Target model: Qwen3-8B GGUF
- Draft model: this EAGLE3 GGUF
- Example settings:
  - `--spec-type draft-eagle3`
  - `--spec-draft-n-max 4`
  - `--spec-draft-p-min 0.5`

Local benchmark observations may vary depending on GPU, quantization, context length, batch size, and prompt type.

## Benchmark Notes

In local testing, EAGLE3 showed stronger gains on structured or predictable outputs such as:

- code generation
- JSON output
- repeated pattern generation
- short factual responses

It was less effective on some tasks such as translation, creative writing, and long reasoning, where draft acceptance may be lower.

## Conversion

Converted with llama.cpp `convert_hf_to_gguf.py` using the original speculator repository and the matching target model directory.

Example conversion command:

```bash
python convert_hf_to_gguf.py \
  RedHatAI/Qwen3-8B-speculator.eagle3 \
  --outtype f16 \
  --target-model-dir Qwen/Qwen3-8B \
  --outfile Qwen3-8B-speculator.eagle3-F16.gguf
```

PowerShell example:

```powershell
python .\convert_hf_to_gguf.py `
  "E:\OLLAMA_MODELS\Qwen3-8B-speculator.eagle3" `
  --outtype f16 `
  --target-model-dir "E:\OLLAMA_MODELS\Qwen3-8B" `
  --outfile "E:\OLLAMA_MODELS\Qwen3-8B-speculator.eagle3-F16.gguf"
```

## Credits

Original EAGLE3 speculator model by RedHatAI:

- `RedHatAI/Qwen3-8B-speculator.eagle3`

Target model:

- `Qwen/Qwen3-8B`

GGUF support and runtime:

- `ggml-org/llama.cpp`

## License

This repository is a converted GGUF version of the original speculator model. The original model license and usage terms apply. Please refer to the upstream repositories for full license details.