---
license: apache-2.0
base_model:
  - deepreinforce-ai/Ornith-1.0-35B
  - AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4
base_model_relation: quantized
library_name: llama.cpp
tags:
  - gguf
  - llama.cpp
  - nvfp4
  - fp4
  - modelopt
  - mtp
  - embedded-mtp
  - speculative-decoding
  - qwen3.6
  - qwen
  - moe
  - experimental
pipeline_tag: text-generation
---

# Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4-MTP-GGUF

Experimental GGUF conversion of **AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4** with an embedded native Qwen3.6 MTP block.

> **This is a self-contained NVFP4 GGUF with embedded native MTP tensors.**
>
> It does not require an external MTP-only draft model, but this embedded-MTP path is experimental.

## Status

**Experimental.**

The model loads and performs native MTP speculative decoding in llama.cpp. However, on the tested system, using the same MTP tensors through a standalone MTP-only GGUF via `--model-draft` was slightly faster than embedding the tensors directly into the GGUF.

For most users, the standard NVFP4 GGUF plus external `--model-draft` is recommended.

## Model Details

* **Source model:** `AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4`
* **Base model:** `deepreinforce-ai/Ornith-1.0-35B`
* **Format:** GGUF
* **Quantization:** NVIDIA NVFP4
* **Architecture:** Qwen3.6 Mixture-of-Experts with hybrid attention
* **MTP:** Embedded native Qwen3.6 MTP block
* **Parameters:** 35B total, approximately 3B activated per token
* **Purpose:** Local inference and native MTP speculative decoding with llama.cpp

## Embedded MTP

This model was created by grafting the native Qwen3.6 MTP block into the converted NVFP4 GGUF.

The resulting GGUF contains:

```text
qwen35moe.block_count = 41
qwen35moe.nextn_predict_layers = 1
```

Additional tensors:

```text
blk.40.*
```

Only the native MTP block was added. The original transformer layers `blk.0.*` through `blk.39.*` were not modified.

Recommended graft source:

[Qwen3.6-35B-A3B-MTP-ONLY-Q6_K.gguf](https://huggingface.co/a4lg/Qwen3.6-35B-A3B-MTP-ONLY-GGUF)

## Compatibility

A recent version of **llama.cpp** with Qwen3.6 MoE, NVFP4, and native MTP support is required.

Tested with:

* Windows
* NVIDIA GeForce RTX 5070 Ti 16 GB
* NVIDIA GeForce RTX 5060 Ti 16 GB
* llama.cpp CUDA backend
* Embedded native Qwen3.6 MTP speculative decoding

Older llama.cpp builds may fail to recognize the `nvfp4` tensor type, may not correctly load associated scale tensors, or may lack compatible Qwen3.6 MoE/MTP support.

Performance may vary with llama.cpp build, GPU split, context size, KV-cache format, prompt, sampling settings, and other runtime options.

## Usage

### llama-server

```bash
llama-server \
  -m Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4-MTP.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --flash-attn on \
  -ngl 99
```

### llama-cli

```bash
llama-cli \
  -m Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4-MTP.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --flash-attn on \
  -ngl 99
```

### Multi-GPU example

```bash
llama-server \
  -m Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4-MTP.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --split-mode layer \
  --tensor-split 1,1 \
  --flash-attn on \
  -ngl 99
```

The best tensor split depends on available VRAM, GPU speed, PCIe topology, context length, and KV-cache placement. An even split is only a starting point.

## Runtime Recommendation

Although the embedded MTP GGUF functions correctly, local benchmarks showed that using the standalone MTP-only GGUF via `--model-draft` was slightly faster than embedding the same tensors directly into the GGUF.

Recommended for most users:

```bash
llama-server \
  -m Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4.gguf \
  --model-draft Qwen3.6-35B-A3B-MTP-ONLY-Q6_K.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3
```

Use this embedded-MTP GGUF mainly if you want a self-contained file or want to experiment with native MTP grafting.

## Benchmark Summary

Local mixed-task benchmark:

| Mode | Aggregate acceptance | Wall time | Notes |
|---|---:|---:|---|
| No MTP | n/a | **21.88 s** | Baseline (standard decoding) |
| External MTP-only (`--model-draft`, Q6_K) | **95.09%** | **22.33 s** | Recommended; highest acceptance and best speculative decoding performance in local tests |
| Embedded MTP | **93.17%** | **24.51 s** | Functional, but slower than the external draft model in local tests |

The embedded model works correctly, but `--model-draft` currently appears to be the better runtime path on the tested setup.

Results may vary depending on llama.cpp version, CUDA graph behavior, GPU offload, batch/ubatch settings, context length, quantization, and prompts.

## Suggested MTP Settings

A reasonable general-purpose starting point is:

```text
--spec-draft-n-max 3
```

For translation, role-play, creative writing, conversational output, or open-ended explanations, start with:

```text
--spec-draft-n-max 2
```

For JSON, fixed templates, repeated patterns, and deterministic code completion, try:

```text
--spec-draft-n-max 4
```

## Grafting Method

The embedded MTP block was generated by grafting the standalone MTP-only GGUF.

This demonstrates that MTP-only GGUF files can serve two purposes:

1. Standalone draft model via `--model-draft`
2. Graft source for creating a self-contained GGUF

The graft process updates:

```text
qwen35moe.block_count: 40 -> 41
qwen35moe.nextn_predict_layers: 1
```

and appends:

```text
blk.40.*
```

It does not modify:

```text
tokenizer
vocabulary
chat template
RoPE settings
original transformer layers
```

## Verification

Check the embedded MTP metadata and tensors:

```powershell
python .\gguf-py\gguf\scripts\gguf_dump.py Qwen3.6-35B-A3B-NVFP4-MTP.gguf |
  Select-String "block_count|nextn_predict_layers|blk\.40"
```

Expected output should include:

```text
qwen35moe.block_count = 41
qwen35moe.nextn_predict_layers = 1
blk.40.attn_k.weight
blk.40.nextn.eh_proj.weight
blk.40.nextn.shared_head_norm.weight
```

## Notes

* This GGUF preserves NVIDIA's NVFP4 tensor type; it is not equivalent to `Q4_K_M`, `Q4_K_S`, or an Unsloth dynamic quant.
* The embedded MTP block is experimental and may not be faster than using `--model-draft`.
* The model is a 35B-total-parameter MoE with approximately 3B parameters activated per token; runtime memory requirements still depend on the full stored checkpoint rather than only the active parameter count.
* NVIDIA's source checkpoint was prepared for ModelOpt and vLLM. llama.cpp support is a separate community implementation and may behave differently from NVIDIA's reference runtime.
* Native MTP accelerates token generation but does not improve prompt-prefill speed in the same way.
* Very short benchmark outputs are more sensitive to run-to-run variance.
* Results are specific to the tested hardware, llama.cpp build, prompts, runtime options, and context configuration.
* Although standard decoding completed slightly faster on this short benchmark, native MTP enables speculative decoding and substantially increases token generation throughput (up to ~115 tok/s in these tests). Acceptance rate is therefore a more meaningful metric than total wall time when evaluating MTP effectiveness.

## Related Projects

* **Standard NVFP4 GGUF:** `AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4`
* **Grafting utility:** `gguf-graft-mtp`

## Credits

* **Qwen Team / Alibaba Cloud** — Qwen3.6-35B-A3B
* **deepreinforce-ai / Ornith-1.0-35B** — Qwen3.6-35B-A3B
* **NVIDIA** — ModelOpt and the original NVFP4 checkpoint
* **ggml-org** — llama.cpp, GGUF, NVFP4 inference support, and native MTP support
* **a4lg** — MTP-only GGUF subset used as a graft source

## License

The source model is distributed under the **Apache License 2.0**.

Users should review the upstream `AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4`, `deepreinforce-ai/Ornith-1.0-35B`, and the MTP-only GGUF source model cards before redistribution or commercial use.