---
license: apache-2.0
base_model: ProCreations/grug-3b
tags:
- grug
- gguf
- llama.cpp
- reasoning
- token-efficient
language:
- en
pipeline_tag: text-generation
---

# grug-3b-gguf

gguf of [ProCreations/grug-3b](https://huggingface.co/ProCreations/grug-3b).

grug think in grug inside `<think>`, then answer normal english. think short for
small question (13 token median), long for hard one (737 token median).

| file | size |
|---|---|
| grug-3b-f16.gguf | 8.34 GB |
| grug-3b-Q8_0.gguf | 4.43 GB |
| grug-3b-Q6_K.gguf | 3.42 GB |
| grug-3b-Q5_K_M.gguf | 2.99 GB |
| grug-3b-Q4_K_M.gguf | 2.57 GB |
| grug-3b-Q3_K_M.gguf | 2.17 GB |

want q4 that hold up better? grug also train one q4-aware:
[ProCreations/grug-3b-qat-q4-gguf](https://huggingface.co/ProCreations/grug-3b-qat-q4-gguf).

## llama.cpp support

Nanbeige4.2 not in upstream llama.cpp yet (issue
[#26086](https://github.com/ggml-org/llama.cpp/issues/26086)). Nanbeige team PR
[#25994](https://github.com/ggml-org/llama.cpp/pull/25994) add it - weight-shared
depth loop, `num_loops=2`. until merge, build from that branch:

```bash
git clone --depth 1 --branch nanbeige42 https://github.com/Nanbeige/llama.cpp
cd llama.cpp && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j
./build/bin/llama-cli -m grug-3b-Q4_K_M.gguf -p "What is 12 times 12?"
```

these gguf converted and load-probed with that branch.