Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,93 @@
+---
+license: other
+license_name: nvidia-open-model-license
+license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
+base_model: nvidia/Nemotron-Cascade-2-30B-A3B
+tags:
+  - nemotron
+  - moe
+  - mamba
+  - gguf
+  - attn-repeat
+  - rys
+quantized_by: audreyt
+pipeline_tag: text-generation
+---
+
+# Nemotron-Cascade-2-30B-A3B with Attention Repeat (Layer 5) - GGUF Q4_K_M
+
+A modified GGUF of [NVIDIA Nemotron-Cascade-2-30B-A3B](https://huggingface.co/nvidia/Nemotron-Cascade-2-30B-A3B) that applies the **RYS (Repeat Your Steps)** layer duplication technique from [shi3z/nemotron-cascade-2-attn-repeat-L5](https://huggingface.co/shi3z/nemotron-cascade-2-attn-repeat-L5), but in GGUF format for use with **ollama** and **llama.cpp**.
+
+## What changed
+
+Layer 5 (the first GQA Attention layer) is physically duplicated in the GGUF. The model goes from 52 to **53 layers**, with layers 5 and 6 being identical attention blocks. All subsequent layers are shifted by 1.
+
+This is the GGUF equivalent of shi3z's `BlockRepeatWrapper` approach. No weights were modified -- only duplicated.
+
+### Per-layer metadata
+
+The GGUF includes correct 53-element per-layer arrays for `attention.head_count_kv` and `feed_forward_length`, so ollama correctly classifies each layer (Mamba2 / GQA Attention / MoE).
+
+## Architecture
+
+53 layers total (was 52):
+- **Mamba-2**: 23 layers (SSM-based, sequential)
+- **MoE**: 23 layers (128 routed experts + 1 shared, top-6)
+- **GQA Attention**: **7 layers** (was 6) at positions 5, **6**, 13, 20, 27, 34, 43
+
+Parameters: 31.6B total, ~3B active per token. Quantization: Q4_K_M.
+
+## Performance
+
+Benchmarked on NVIDIA DGX Spark (GB10 Blackwell, 120GB unified LPDDR5X):
+
+| Metric | Base cascade-2 | attn-repeat-L5 |
+|--------|---------------|-----------------|
+| Generation speed | 74.5 tok/s | 62.5 tok/s |
+| Prompt eval | 81.7 tok/s | 299.0 tok/s |
+| Model size | 23 GB | 22.6 GB |
+| VRAM loaded | 26 GB | 27 GB |
+
+~16% slower generation, ~3.7x faster prompt processing. Per shi3z's benchmarks, +6.7 percentage points on BBH-style reasoning.
+
+## Usage with ollama
+
+```bash
+# Download and create model
+ollama create nemotron-cascade-2-attn-repeat -f Modelfile
+
+# Run
+ollama run nemotron-cascade-2-attn-repeat "Hello!"
+```
+
+**Modelfile:**
+```
+FROM nemotron-cascade-2-attn-repeat-L5-Q4_K_M.gguf
+TEMPLATE {{ .Prompt }}
+RENDERER nemotron-3-nano
+PARSER nemotron-3-nano
+PARAMETER temperature 1
+PARAMETER top_p 0.95
+```
+
+## How this GGUF was built
+
+The GGUF was produced by a Python script that:
+
+1. Reads the original `nemotron-cascade-2` Q4_K_M GGUF from ollama
+2. Duplicates all 5 tensors of block 5 (attn_q, attn_k, attn_v, attn_output, attn_norm) as block 6
+3. Shifts all blocks >= 6 up by 1
+4. Updates `block_count` from 52 to 53
+5. Rebuilds the per-layer `attention.head_count_kv` and `feed_forward_length` arrays with the inserted entry
+
+Source: built on DGX Spark running ollama 0.20.0, using gguf-py 0.18.0.
+
+## Credits
+
+- **NVIDIA** for [Nemotron-Cascade-2-30B-A3B](https://huggingface.co/nvidia/Nemotron-Cascade-2-30B-A3B)
+- **shi3z** for the [attn-repeat-L5 discovery](https://huggingface.co/shi3z/nemotron-cascade-2-attn-repeat-L5) and RYS technique
+- Built with assistance from Claude Code (Anthropic)
+
+## License
+
+NVIDIA Open Model License (inherited from base model).