---
license: apache-2.0
base_model: xunkutech-ai/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp
base_model_relation: quantized
language:
- en
library_name: mlx
pipeline_tag: text-generation
tags:
- mlx
- mlx-lm
- mlx-vlm
- safetensors
- oQ4
- omlx
- qwen3_5
- qwen3.5
- reasoning
- long-context
- 1M-context
- function-calling
- tool-use
- agentic
- cybersecurity
- biomedical
- conversational
- mtp
---

# Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ4-mtp

An **oQ4 quantized** MLX version of [Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp](https://huggingface.co/xunkutech-ai/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp) with integrated **Multi-Token Prediction (MTP)** head for accelerated speculative decoding on Apple Silicon.

Quantized with **oMLX oQ4** — a smart mixed-precision quantization that selectively preserves critical layers while compressing the rest for significant memory savings with minimal quality loss.

## Model Summary

- **Format:** MLX safetensors, oQ4 quantized
- **Precision:** Mixed (oQ4 smart quantization)
- **Parameters:** ~9B total
- **Context length:** 1,048,576 tokens
- **Architecture:** Qwen3.5-style hybrid attention text model with MTP head
- **MTP layers:** 1 (embedded, `mtp_num_hidden_layers: 1`)
- **Primary use:** local text generation and reasoning on Apple Silicon with MTP-accelerated decoding
- **Base model:** `xunkutech-ai/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp`

## Compatibility Notice

> **[oMLX](https://github.com/jundot/omlx)** is required to load this model with MTP support.
> Standard `mlx-vlm`, `mlx-lm`, and **LM Studio do NOT support MTP** and will fail with:
> `ValueError: Received 15 parameters not in model: language_model.mtp.*`.

## Installation

```bash
pip install -U git+https://github.com/jundot/omlx.git
```

## Usage with oMLX

```bash
# Start server with MTP draft-mode
omlx serve --model /path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ4-mtp
```

```python
from omlx import load, generate

model_id = "/path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ4-mtp"
model, tokenizer = load(model_id)

result = generate(
    model=model,
    tokenizer=tokenizer,
    prompt="Explain speculative decoding with MTP.",
    max_tokens=2048,
    temperature=0.6,
)
print(result)
```

## Files

| File | Size | Description |
|------|------|-------------|
| `config.json` | ~22 KB | oQ4 quantized config with `mtp_num_hidden_layers: 1` |
| `model-00001-of-00002.safetensors` | ~5.0 GB | Quantized shard 1/2 |
| `model-00002-of-00002.safetensors` | ~1.2 GB | Quantized shard 2/2 (+ MTP head) |
| `tokenizer.json` | ~20 MB | Qwen3.5 tokenizer |
| `chat_template.jinja` | ~8 KB | Qwythos chat template |

## Recommended Sampling

```python
generation_kwargs = {
    "temperature": 0.6,
    "top_p": 0.95,
    "top_k": 20,
    "repetition_penalty": 1.05,
    "max_tokens": 4096,
}
```

## License

Released under the same license as the upstream model: Apache-2.0.

## Acknowledgements

- Original model: [empero-ai/Qwythos-9B-Claude-Mythos-5-1M](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M)
- Original developers: Empero AI
- Base family: Qwen3.5
- MTP support: [oMLX](https://github.com/jundot/omlx)
- MLX ecosystem: Apple MLX