---
license: other
license_name: nvidia-open-model-license
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
base_model: nvidia/Nemotron-Cascade-2-30B-A3B
tags:
  - nemotron
  - moe
  - mamba
  - gguf
  - attn-repeat
  - rys
quantized_by: audreyt
pipeline_tag: text-generation
---

# Nemotron-Cascade-2-30B-A3B with Attention Repeat (Layer 5) - GGUF Q4_K_M

A modified GGUF of [NVIDIA Nemotron-Cascade-2-30B-A3B](https://huggingface.co/nvidia/Nemotron-Cascade-2-30B-A3B) that applies the **RYS (Repeat Your Steps)** layer duplication technique from [shi3z/nemotron-cascade-2-attn-repeat-L5](https://huggingface.co/shi3z/nemotron-cascade-2-attn-repeat-L5), but in GGUF format for use with **ollama** and **llama.cpp**.

## What changed

Layer 5 (the first GQA Attention layer) is physically duplicated in the GGUF. The model goes from 52 to **53 layers**, with layers 5 and 6 being identical attention blocks. All subsequent layers are shifted by 1.

This is the GGUF equivalent of shi3z's `BlockRepeatWrapper` approach. No weights were modified -- only duplicated.

### Per-layer metadata

The GGUF includes correct 53-element per-layer arrays for `attention.head_count_kv` and `feed_forward_length`, so ollama correctly classifies each layer (Mamba2 / GQA Attention / MoE).

## Architecture

53 layers total (was 52):
- **Mamba-2**: 23 layers (SSM-based, sequential)
- **MoE**: 23 layers (128 routed experts + 1 shared, top-6)
- **GQA Attention**: **7 layers** (was 6) at positions 5, **6**, 13, 20, 27, 34, 43

Parameters: 31.6B total, ~3B active per token. Quantization: Q4_K_M.

## Performance

Benchmarked on NVIDIA DGX Spark (GB10 Blackwell, 120GB unified LPDDR5X):

| Metric | Base cascade-2 | attn-repeat-L5 |
|--------|---------------|-----------------|
| Generation speed | 74.5 tok/s | 62.5 tok/s |
| Prompt eval | 81.7 tok/s | 299.0 tok/s |
| Model size | 23 GB | 22.6 GB |
| VRAM loaded | 26 GB | 27 GB |

~16% slower generation, ~3.7x faster prompt processing. Per shi3z's benchmarks, +6.7 percentage points on BBH-style reasoning.

## Usage with ollama

```bash
# Download and create model
ollama create nemotron-cascade-2-attn-repeat -f Modelfile

# Run
ollama run nemotron-cascade-2-attn-repeat "Hello!"
```

**Modelfile:**
```
FROM nemotron-cascade-2-attn-repeat-L5-Q4_K_M.gguf
TEMPLATE {{ .Prompt }}
RENDERER nemotron-3-nano
PARSER nemotron-3-nano
PARAMETER temperature 1
PARAMETER top_p 0.95
```

## How this GGUF was built

The GGUF was produced by a Python script that:

1. Reads the original `nemotron-cascade-2` Q4_K_M GGUF from ollama
2. Duplicates all 5 tensors of block 5 (attn_q, attn_k, attn_v, attn_output, attn_norm) as block 6
3. Shifts all blocks >= 6 up by 1
4. Updates `block_count` from 52 to 53
5. Rebuilds the per-layer `attention.head_count_kv` and `feed_forward_length` arrays with the inserted entry

Source: built on DGX Spark running ollama 0.20.0, using gguf-py 0.18.0.

## Credits

- **NVIDIA** for [Nemotron-Cascade-2-30B-A3B](https://huggingface.co/nvidia/Nemotron-Cascade-2-30B-A3B)
- **shi3z** for the [attn-repeat-L5 discovery](https://huggingface.co/shi3z/nemotron-cascade-2-attn-repeat-L5) and RYS technique
- Built with assistance from Claude Code (Anthropic)

## License

NVIDIA Open Model License (inherited from base model).