---
license: apache-2.0
base_model: Nanbeige/Nanbeige4.2-3B
pipeline_tag: text-generation
library_name: gguf
tags:
  - gguf
  - llama.cpp
  - nanbeige
---

# Nanbeige4.2-3B GGUF

Unofficial GGUF conversions of
[Nanbeige/Nanbeige4.2-3B](https://huggingface.co/Nanbeige/Nanbeige4.2-3B),
pinned to source revision
`451ed48c3273ecef7ea8faaa43c31ce529763bb1`.

Nanbeige4.2-3B is not an ordinary Llama stack. It reuses 22 physical decoder
layers for two loops, applies the shared final RMSNorm at the loop boundary,
and needs 44 logical KV-cache slots. These files therefore use the custom GGUF
architecture name `nanbeige` and require the included llama.cpp patch.

## Files

| File | Size | SHA-256 |
|---|---:|---|
| `Nanbeige4.2-3B-Q4_K_M.gguf` | 2,574,807,840 bytes | `99c7bfb88907f7eee0a04c4314f1c46bca391819478d8cb90b3e164f09576489` |
| `Nanbeige4.2-3B-Q8_0.gguf` | 4,434,787,104 bytes | `32b60cd0fb3da4d8a3c01d0ca7d0461818c29c81ffb42f8fc71a94141bd803c6` |
| `llama.cpp-nanbeige.patch` | 10,137 bytes | `1177badb8f8f6228052d2f67a9eabe81f2ce69f3ea0b92cf7b1ea2352204284a` |

The source checkpoint contains 4,169,800,704 physical parameters. The Q4_K_M
file is about 4.93 bits per weight; Q8_0 is about 8.51 bits per weight.

## Runtime

Apply the patch to the exact tested llama.cpp revision:

```bash
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout cf512566dc89d7a2bdefd43f6d4b85a1c18e93a7
git apply /path/to/llama.cpp-nanbeige.patch
cmake -B build
cmake --build build --config Release
```

Then run either quant:

```bash
./build/bin/llama-cli \
  -m Nanbeige4.2-3B-Q4_K_M.gguf \
  -c 4096 -n 256 \
  -p "Hello"
```

Stock llama.cpp builds that do not include Nanbeige support will reject the
`nanbeige` architecture. The patch adds conversion metadata, tensor loading,
loop-boundary normalization, and 44-layer cache handling; it does not replace
model weights.

## Validation

Built and tested on Windows with llama.cpp
`cf512566dc89d7a2bdefd43f6d4b85a1c18e93a7` and an AMD Ryzen 9 7950X3D.

- Both files loaded as architecture `nanbeige`, block count 44, loop count 2.
- Q4_K_M: prompt benchmark 92.77 tok/s; generation benchmark 15.67 tok/s.
- Q8_0: prompt benchmark 53.95 tok/s; generation benchmark 9.37 tok/s.
- Deterministic eight-token generation completed for both files.
- Interactive Q8_0 sample measured 68.6 tok/s prompt and 10.2 tok/s generation.

These are CPU smoke/throughput tests, not quality benchmarks. The original
model advertises a 262,144-token context, but local validation used a short
512-token context.

## License

The base model is Apache-2.0 licensed. Review the upstream model card for its
full terms and intended-use information.