---
license: apache-2.0
base_model: Ornith-1.0-35B
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
tags:
  - the-rock8
  - rock8
  - rdna4
  - gfx1201
  - radeon-ai-pro-r9700
  - rx-9070
  - fp8
  - f8e4m3
  - quark
  - gguf
  - amd
  - rocm
  - llama.cpp
  - quantized
  - moe
  - mixture-of-experts
model_name: Quacken-Ornith-35B-FP8
quantized_by: The Rock8
---

# Quacken-Ornith-35B-FP8
### The Rock8 - Got any weights? 💪🦆

Native **fp8 E4M3** GGUF of **Ornith-1.0-35B** - a Mixture-of-Experts
GatedDeltaNet-hybrid (multimodal-capable) model - for **AMD RDNA4** (gfx1201 -
Radeon AI PRO R9700 / RX 9070 / 9070 XT / W-series), quantized with **AMD Quark**
from the full-precision BF16 weights by [The Rock8](https://github.com/The-Monk/The-Rock8).

The Rock8's llama.cpp fork runs this fp8 on RDNA4's native WMMA fp8 tensor cores -
no dequant-to-f16 fallback.

## What it is
- **Format:** fp8 E4M3 (`F8E4M3`), block-scaled, produced by AMD Quark from BF16.
- **Architecture:** MoE + GatedDeltaNet SSM-hybrid. **No MTP head** (the checkpoint
  ships zero MTP tensors) - so self-speculative `--spec-type draft-mtp` is not available
  for this model; run it as a plain target.
- **Target:** AMD RDNA4 / gfx1201; ~38 GB -> runs **2-GPU** (tensor-split across two
  32 GB R9700s) or one large card.
- **Runtime:** The Rock8 (llama.cpp fork with native RDNA4 fp8 kernels) on TheRock ROCm 7.13.
- **File:** `ornith-1.0-35b-quark-fp8-authentic.gguf` (37.8 GB).

## Important run note
This is an SSM-hybrid architecture. Run it with **`llama-server`** or
**`llama-perplexity`** - **not `llama-bench`** (llama-bench cannot allocate the
SSM-hybrid context and will fail on this arch). That is a tooling limitation, not a
model defect; the model is validated and generates correctly under llama-server.

## Source model + license
- **Source:** Ornith-1.0-35B.
- **License:** **Apache-2.0** (redistribution of this quantized derivative is permitted
  with attribution). This is a derivative work.

## Validation (real gfx1201 hardware)
| Metric | Value |
|---|---:|
| Perplexity (wikitext, 20 chunks, n_ctx=512), 2-GPU | **6.7010** |

Validated via `llama-perplexity` on dual R9700 (gfx1201).

## Run it (2-GPU)
```bash
# use llama-server (NOT llama-bench); -ngl 999 tensor-splits across both R9700s
llama-server -m ornith-1.0-35b-quark-fp8-authentic.gguf -ngl 999 --host 0.0.0.0 --port 13305

curl -s http://localhost:13305/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"What do you call a dried grape? Answer in one word."}],"max_tokens":16}'
# expect: raisin
```

### Lemonade appliance (container)
```bash
podman run -d --rm --runtime crun --name lemonade \
  --device /dev/kfd --device /dev/dri \
  --group-add keep-groups --security-opt seccomp=unconfined \
  -v /path/to/quacken-ornith-35b:/models:ro \
  -e MODEL=/models/ornith-1.0-35b-quark-fp8-authentic.gguf -e MODEL_NAME=Quacken-Ornith-35B-FP8 \
  -p 13305:13305 \
  ghcr.io/the-monk/the-rock8:rdna4-tr713 serve
# ~38 GB - needs both GPUs; do NOT pin HIP_VISIBLE_DEVICES to a single card
```
Container (same image on each registry; `--runtime crun` is required for GPU):
`ghcr.io/the-monk/the-rock8:rdna4-tr713` - `docker.io/gorilla4x/the-rock8:rdna4-tr713` - `quay.io/the-monk/the-rock8:rdna4-tr713`
(images may not be pushed to every registry yet).

## The Rock8 - RDNA4 fp8 (links)
- **GitHub:** [The-Rock8](https://github.com/The-Monk/The-Rock8) - kernels, patch series, appliance recipe, full feature doc.
- **Collection:** [The Rock8 - RDNA4 fp8](https://huggingface.co/collections/Gorilla4X/the-rock8-rdna4-fp8-6a547070f667cb41db0bc2ed).
- **Sibling models:**
  [Quacken-8B-FP8](https://huggingface.co/Gorilla4X/Quacken-8B-FP8) -
  [Quacken-R1-14B-FP8](https://huggingface.co/Gorilla4X/Quacken-R1-14B-FP8) -
  [Quacken-27B-FP8](https://huggingface.co/Gorilla4X/Quacken-27B-FP8) -
  [Quacken-35B-A3B-FP8](https://huggingface.co/Gorilla4X/Quacken-35B-A3B-FP8).

*Every artifact links to the others - land on any one, reach them all.*