Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ4-mtp

An oQ4 quantized MLX version of Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp with integrated Multi-Token Prediction (MTP) head for accelerated speculative decoding on Apple Silicon.

Quantized with oMLX oQ4 — a smart mixed-precision quantization that selectively preserves critical layers while compressing the rest for significant memory savings with minimal quality loss.

Model Summary

• Format: MLX safetensors, oQ4 quantized
• Precision: Mixed (oQ4 smart quantization)
• Parameters: ~9B total
• Context length: 1,048,576 tokens
• Architecture: Qwen3.5-style hybrid attention text model with MTP head
• MTP layers: 1 (embedded, mtp_num_hidden_layers: 1)
• Primary use: local text generation and reasoning on Apple Silicon with MTP-accelerated decoding
• Base model: xunkutech-ai/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp

Compatibility Notice

oMLX is required to load this model with MTP support. Standard mlx-vlm, mlx-lm, and LM Studio do NOT support MTP and will fail with: ValueError: Received 15 parameters not in model: language_model.mtp.*.

Installation

pip install -U git+https://github.com/jundot/omlx.git

Usage with oMLX

# Start server with MTP draft-mode
omlx serve --model /path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ4-mtp

from omlx import load, generate

model_id = "/path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ4-mtp"
model, tokenizer = load(model_id)

result = generate(
    model=model,
    tokenizer=tokenizer,
    prompt="Explain speculative decoding with MTP.",
    max_tokens=2048,
    temperature=0.6,
)
print(result)

Files

File	Size	Description
config.json	~22 KB	oQ4 quantized config with mtp_num_hidden_layers: 1
model-00001-of-00002.safetensors	~5.0 GB	Quantized shard 1/2
model-00002-of-00002.safetensors	~1.2 GB	Quantized shard 2/2 (+ MTP head)
tokenizer.json	~20 MB	Qwen3.5 tokenizer
chat_template.jinja	~8 KB	Qwythos chat template

Recommended Sampling

generation_kwargs = {
    "temperature": 0.6,
    "top_p": 0.95,
    "top_k": 20,
    "repetition_penalty": 1.05,
    "max_tokens": 4096,
}

License

Released under the same license as the upstream model: Apache-2.0.

Acknowledgements

• Original model: empero-ai/Qwythos-9B-Claude-Mythos-5-1M
• Original developers: Empero AI
• Base family: Qwen3.5
• MTP support: oMLX
• MLX ecosystem: Apple MLX