Upload folder using huggingface_hub

.gitattributes

@@ -1,35 +1,11 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tar.gz filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,205 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+base_model:
+  - RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3
+  - unsloth/gemma-4-26B-A4B-it-GGUF
+base_model_relation: quantized
+library_name: llama.cpp
+tags:
+  - gguf
+  - llama.cpp
+  - eagle3
+  - speculative-decoding
+  - speculator
+  - draft-model
+  - gemma-4
+  - gemma
+  - moe
+  - redhatai
+  - unsloth
+pipeline_tag: text-generation
+
+---
+
+# Gemma 4 26B-A4B IT EAGLE3 Speculator GGUF
+
+This repository contains GGUF conversions and quantizations of **RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3** for use with **llama.cpp EAGLE3 speculative decoding**.
+
+> [!IMPORTANT]
+> This is **not a standalone chat model**. It is an **EAGLE3 draft/speculator model** and must be used together with the matching target/verifier model.
+
+* Target model: `unsloth/gemma-4-26B-A4B-it-GGUF`
+* Speculator source: `RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3`
+* Runtime: llama.cpp with `--spec-type draft-eagle3`
+
+## Files
+
+| File                                               | Type                             | Notes                                                         |
+| -------------------------------------------------- | -------------------------------- | ------------------------------------------------------------- |
+| `gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf`    | EAGLE3 speculator GGUF           | Converted from the original RedHatAI safetensors checkpoint   |
+| `gemma-4-26B-A4B-it-speculator.eagle3-Q8_0.gguf`   | Quantized EAGLE3 speculator GGUF | Quantized from the F16 GGUF                                   |
+| `gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf` | Quantized EAGLE3 speculator GGUF | Quantized from the F16 GGUF; may be faster for draft decoding |
+
+## Usage with llama.cpp
+
+Example:
+
+```bash
+llama-server \
+  -m gemma-4-26B-A4B-it-Q4_K_M.gguf \
+  -md gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf \
+  --spec-type draft-eagle3 \
+  --spec-draft-n-max 4 \
+  --spec-draft-p-min 0.5 \
+  -c 32768 \
+  -ngl 99 \
+  -fa on
+```
+
+Windows CMD example:
+
+```cmd
+llama-server.exe ^
+  -m gemma-4-26B-A4B-it-Q4_K_M.gguf ^
+  -md gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf ^
+  --spec-type draft-eagle3 ^
+  --spec-draft-n-max 4 ^
+  --spec-draft-p-min 0.5 ^
+  -c 32768 ^
+  -ngl 99 ^
+  -fa on
+```
+
+PowerShell example:
+
+```powershell
+.\llama-server.exe `
+  -m "gemma-4-26B-A4B-it-Q4_K_M.gguf" `
+  -md "gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf" `
+  --spec-type draft-eagle3 `
+  --spec-draft-n-max 4 `
+  --spec-draft-p-min 0.5 `
+  -c 32768 `
+  -ngl 99 `
+  -fa on
+```
+
+## Important Notes
+
+This GGUF file is only the **draft/speculator model**. You still need a compatible GGUF of the target model, such as `unsloth/gemma-4-26B-A4B-it-GGUF`.
+
+Do **not** use this speculator with unrelated models such as Gemma 4 12B, Gemma 4 31B, Gemma 3, Qwen, Llama, Mistral, or other non-matching models. EAGLE3 speculators are target-specific.
+
+Even small differences in the target model, prompt format, quantization, or runtime settings may affect draft acceptance rate and overall speed.
+
+## Tested Configuration
+
+Tested with:
+
+* Runtime: llama.cpp with EAGLE3 support
+* Target model: `unsloth/gemma-4-26B-A4B-it-GGUF`
+* Draft model: this EAGLE3 GGUF
+* Example settings:
+
+  * `--spec-type draft-eagle3`
+  * `--spec-draft-n-max 4`
+  * `--spec-draft-p-min 0.5`
+
+Local benchmark observations may vary depending on GPU, quantization, context length, batch size, sampling settings, and prompt type.
+
+## Benchmark Notes
+
+In local testing, Gemma 4 26B-A4B IT without EAGLE3 already showed strong baseline decoding speed.
+
+With EAGLE3 enabled, the draft acceptance rate was around `0.70` in local testing, with stronger gains on structured or predictable tasks such as:
+
+* JSON output
+* stepwise math
+* code completion
+* summarization
+* long reasoning
+* repeated pattern generation
+
+It was less effective on some open-ended or language-sensitive tasks such as:
+
+* translation
+* creative writing
+* general explanation
+* some factual QA prompts
+
+On this model, EAGLE3 may be useful for structured output, agent/tool-style responses, code completion, and predictable formats. For general chat, translation, roleplay, or creative writing, the non-speculative baseline may be competitive or more consistent.
+
+On smaller VRAM setups, the extra draft/speculator model may reduce the practical benefit of EAGLE3. In those cases, native MTP models or the base Gemma 4 26B-A4B model without speculative decoding may be more efficient.
+
+## Conversion
+
+Converted with llama.cpp `convert_hf_to_gguf.py` using the original speculator repository and the matching target model directory.
+
+Example conversion command:
+
+```bash
+python convert_hf_to_gguf.py \
+  RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3 \
+  --outtype f16 \
+  --target-model-dir gemma-4-26B-A4B-it \
+  --outfile gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf
+```
+
+PowerShell example:
+
+```powershell
+python .\convert_hf_to_gguf.py `
+  "E:\OLLAMA_MODELS\gemma-4-26B-A4B-it-speculator.eagle3" `
+  --outtype f16 `
+  --target-model-dir "E:\OLLAMA_MODELS\gemma-4-26B-A4B-it" `
+  --outfile "E:\OLLAMA_MODELS\gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf"
+```
+
+## Quantization
+
+The F16 GGUF can be quantized with `llama-quantize`.
+
+Q8_0 example:
+
+```bash
+llama-quantize \
+  gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf \
+  gemma-4-26B-A4B-it-speculator.eagle3-Q8_0.gguf \
+  Q8_0
+```
+
+Q4_K_M example:
+
+```bash
+llama-quantize \
+  gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf \
+  gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf \
+  Q4_K_M
+```
+
+PowerShell example:
+
+```powershell
+.\llama-quantize.exe `
+  "E:\OLLAMA_MODELS\gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf" `
+  "E:\OLLAMA_MODELS\gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf" `
+  Q4_K_M
+```
+
+## Credits
+
+Original EAGLE3 speculator model by RedHatAI:
+
+* `RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3`
+
+Target GGUF model:
+
+* `unsloth/gemma-4-26B-A4B-it-GGUF`
+
+GGUF support and runtime:
+
+* `ggml-org/llama.cpp`
+
+## License
+
+This repository is a converted GGUF version of the original speculator model. The original model license and usage terms apply. Please refer to the upstream repositories for full license details.

gemma-4-26B-A4B-it-speculator.eagle3-F16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7435083b2a684a77cfef833eb91c43d584fb36e15a12c734399b8048578e04e
+size 1862057504

gemma-4-26B-A4B-it-speculator.eagle3-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a7c314a10444e9c45c970d6aaa1e2541d211fa969e14dab938b8acaf1e0915d
+size 557793312

gemma-4-26B-A4B-it-speculator.eagle3-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e4eb3e400608378b89a66a3af5707539f54ff64571697af1a8b3779bdd52a24
+size 992420384