---
license: apache-2.0
tags:
- gguf
- qwen3.5
- openmythos
- build-small-hackathon
datasets:
- build-small-hackathon/CVE_Vulnerailities_Detailed
- himanshu17HF/ArvixImport-Filtered-Final
base_model:
- build-small-hackathon/OpenMythos
- Qwen/Qwen3.6-27B
---

# OpenMythos 27B - GGUF

GGUF quantisation of [build-small-hackathon/OpenMythos](https://huggingface.co/build-small-hackathon/OpenMythos),
a fine-tune of [Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B).

Converted with `convert_hf_to_gguf.py --no-mtp` from llama.cpp build 9658.
The fine-tune does not include MTP head weights (dropped during training), so MTP
is not available in this GGUF.

## Available Quantisations

| File | Size | Type |
|------|------|------|
| OpenMythos-27B-F16.gguf | 53.8 GB | F16 |
| OpenMythos-27B-Q5_K.gguf | 18.3 GB | Q5_K_M |
| OpenMythos-27B-Q4_K.gguf | 15.4 GB | Q4_K_M |
| OpenMythos-27B-Q6_K.gguf | 21.2 GB | Q6_K |

## Benchmark

Evaluated with [SecEval](https://github.com/XuanwuAI/SecEval) (commit 7aef317) on 2189
multiple-choice security questions. Backend: llama.cpp OpenAI-compatible server, fully
offloaded to GPU. No chain-of-thought / reasoning enabled (`enable_thinking=false`).

Prompt formatted with a system prompt requesting letter-only answers (no explanation).

| Set | Model | Score |
|-----|-------|-------|
| A | OpenMythos-27B-Q5_K | 1703 / 2189 (77.8%) |
| B | VulnLLM-R-7B | 1315 / 2189 (60.1%) |

### OpenMythos-27B-Q5_K test parameters

- model: `OpenMythos-27B-Q5_K.gguf`
- inference: `temp=0.2`, `top_p=0.8`, `top_k=20`, `min_p=0.05`, `repeat_penalty=1.02`
- benchmark script: `/mnt/storage/SecEval-tmp/run_bench.py`
- output: `seceval-1781809723.json`
- prompt speed: 282 tok/s | generation speed: 68 tok/s

#### Per-topic scores

| Topic | Score |
|-------|-------|
| PenTest | 84.2% |
| MemorySafety | 83.3% |
| WebSecurity | 82.7% |
| Vulnerability | 77.8% |
| NetworkSecurity | 77.4% |
| SoftwareSecurity | 75.0% |
| ApplicationSecurity | 74.8% |
| SystemSecurity | 73.6% |
| Cryptography | 71.4% |

### VulnLLM-R-7B test parameters

- model: `VulnLLM-R-7B.Q6_K.gguf`
- inference: same settings as above
- output: `seceval-1781811525.json`
- prompt speed: 148 tok/s | generation speed: 39 tok/s

#### Per-topic scores

| Topic | Score |
|-------|-------|
| PenTest | 70.9% |
| WebSecurity | 66.4% |
| Vulnerability | 58.7% |
| NetworkSecurity | 58.3% |
| SystemSecurity | 56.4% |
| SoftwareSecurity | 54.7% |
| ApplicationSecurity | 54.7% |
| MemorySafety | 54.2% |
| Cryptography | 28.6% |

Full detailed results are included in this repo: `seceval-1781809723.json` and
`seceval-1781811525.json`.

## Usage

### llama-server (recommended)

```ini
[OpenMythos-27B]
model = /mnt/storage/models/OpenMythos/OpenMythos-27B-Q5_K.gguf
chat-template-file = /mnt/storage/llama-server/chat_template-v15.jinja
ctx-size = 65536
cache-type-k = q8_0
cache-type-v = q8_0
cache-prompt = on
cache-reuse = 2048
batch-size = 4096
ubatch-size = 4096
kv-unified = on
parallel = 1
gpu-layers = all
temp = 0.2
top-p = 0.8
top-k = 20
min-p = 0.05
presence-penalty = 0.2
repeat-penalty = 1.02
spec-type = ngram-mod
spec-draft-n-max = 5
reasoning-format = deepseek
swa-checkpoints = 5
```

### llama-cli

```bash
/mnt/storage/llama.cpp/build/bin/llama-cli \
  -m /mnt/storage/models/OpenMythos/OpenMythos-27B-Q5_K.gguf \
  --chat-template-file /mnt/storage/llama-server/chat_template-v15.jinja \
  -c 65536 -b 4096 --ubatch-size 4096 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --kv-unified -t 8 -fa \
  --temp 0.2 --top-p 0.8 --top-k 20 --min-p 0.05 \
  --presence-penalty 0.2 --repeat-penalty 1.02 \
  -ngl all \
  -p "Your prompt here"
```