Qwythos-9B-Claude-Mythos-5-1M-NVFP4-GGUF

NVFP4 GGUF quantization of empero-ai/Qwythos-9B-Claude-Mythos-5-1M -- a full-parameter reasoning model built on a deeply uncensored Qwen3.5-9B base, post-trained on 500M+ tokens of Claude Mythos and Claude Fable traces with chain-of-thought generated in-house by Empero AI's internal tool rethink.

What makes Qwythos special

• 1M token context -- YaRN rope-scaling enabled by default for a full 1,048,576-token context window. One of the longest context windows in any 9B open-weight model. Suitable for whole-codebase reasoning, multi-document research, and long agentic trajectories.
• Massive benchmark gains over base -- +34 pts MMLU, +30 pts gsm8k-strict, +19 pts gsm8k-flex under matched evaluation.
• Native function calling -- OpenAI/Qwen3.5-style tool use out of the box. Pass tools=[...] and the model emits valid <tool_call> blocks. Self-corrects with Python executor and web search (7/7 test prompts succeeded).
• Uncensored by design -- Engages substantively with technically demanding questions across cybersecurity, red-teaming, biology, pharmacology, and clinical medicine where over-aligned models refuse or hedge.
• Reasoning model -- Every answer opens with a <think> block before the final response. Use generous max_new_tokens (16,384 recommended).

Domain strengths

• Cybersecurity -- SQL injection mitigations, TLS handshake structure, EDR/process-injection detection, MITRE ATT&CK ransomware kill chains, hashcat modes, CVE analysis.
• Biomedical -- CRISPR-Cas9 mechanisms, mRNA vaccines, SARS-CoV-2 spike protein, antibiotic resistance, receptor pharmacology, organophosphate AChE inhibition.
• Clinical medicine -- ACS chest-pain differential, type-2 diabetes pathophysiology, sepsis recognition (qSOFA), therapeutic-window reasoning.
• Math -- 86% gsm8k, multi-step word problems, competition math. Verified by Python executor when invoked.

About NVFP4

NVFP4 is NVIDIA's native 4-bit floating point format (E4M3) for Blackwell architecture GPUs.

• Native tensor core acceleration on RTX 50-series
• Better dynamic range than INT4 formats
• No dequantization overhead -- processed directly in FP4

When to use NVFP4 vs other formats:

• NVFP4 -- Best for Blackwell GPUs (RTX 5060 Ti, 5070, 5080, 5090, B100, B200)
• Q4_K_M -- Best for pre-Blackwell GPUs and CPU inference
• MXFP4 -- Open standard, works on any GPU with MX support

Files

File	Type	Size	Description
qwythos-9b-nvfp4.gguf	NVFP4	~5.0 GB	Text model (4.74 BPW)
mmproj-qwythos-9b-f16.gguf	F16	~918 MB	Vision encoder (SigLIP ViT, 27 layers)

Quantization Details

Property	Value
Format	NVFP4 (E4M3)
Bits Per Weight	4.74 BPW
Source Model	empero-ai/Qwythos-9B-Claude-Mythos-5-1M
Architecture	Qwen3_5ForConditionalGeneration
Parameters	9.4B (BF16 source)
Layers	32 (hybrid Gated DeltaNet + full attention)
Hidden Size	4096
Context Length	1,048,576 (1M, YaRN)
Vision	Yes (SigLIP ViT, frozen from base)
Thinking	Enabled by default (opt-out via enable_thinking=false)
Training	500M+ tokens, Claude Mythos/Fable traces, full SFT

Usage

llama.cpp CLI

# Text only
./llama-cli -m qwythos-9b-nvfp4.gguf -p "Hello" -n 100

# With vision (requires mmproj)
./llama-server -m qwythos-9b-nvfp4.gguf \
  --mmproj mmproj-qwythos-9b-f16.gguf \
  --host 0.0.0.0 --port 8080 -ngl 99

llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="qwythos-9b-nvfp4.gguf",
    n_gpu_layers=-1,
    chat_format="chatml"
)

output = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Explain how organophosphate nerve agents inhibit acetylcholinesterase."}],
    max_tokens=4096
)
print(output["choices"][0]["message"]["content"])

huggingface-hub

from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="FreedomAISVR/Qwythos-9B-Claude-Mythos-5-1M-NVFP4-GGUF",
    filename="qwythos-9b-nvfp4.gguf"
)
mmproj_path = hf_hub_download(
    repo_id="FreedomAISVR/Qwythos-9B-Claude-Mythos-5-1M-NVFP4-GGUF",
    filename="mmproj-qwythos-9b-f16.gguf"
)

Sampling recommendations

Qwythos was trained as a reasoning model. Use these settings for best results:

temperature=0.6
top_p=0.95
top_k=20
repetition_penalty=1.05
max_new_tokens=16384

Greedy decoding or very-low-temperature (T<=0.3) can cause repetition loops on long generations.

Quantization Pipeline

1. Download source: empero-ai/Qwythos-9B-Claude-Mythos-5-1M
2. Convert to F16 GGUF: convert_hf_to_gguf.py --outtype f16
3. Extract mmproj: convert_hf_to_gguf.py --mmproj --outtype f16
4. Quantize text: llama-quantize input-f16.gguf output-nvfp4.gguf NVFP4
5. Patch GGUF metadata: block_count 33->32, nextn_predict_layers 1->0

Hardware Requirements

Component	Requirement
GPU	NVIDIA Blackwell (RTX 50-series) for full acceleration
VRAM	~6 GB minimum
RAM	~16 GB recommended
Storage	~6 GB

Limitations

• Reasoning model -- Every answer opens with <think> block. Allow generous token budget.
• Text-only fine-tune -- Vision tower was frozen; vision behavior is inherited from base and was not tuned.
• Uncensored -- Add application-level safety layer for end-user deployments.
• Verify specifics -- Like all 9B models, can over-commit to specific identifiers (CVEs, drug dosages). Pair with tools for accuracy-critical deployments.

License

Apache 2.0 (inherited from Qwen3.5-9B base)

Acknowledgements

• Developed by Empero AI
• Base model: Qwen3.5-9B (Alibaba Qwen team)
• Training: TRL + Transformers
• Linear-attention kernels: flash-linear-attention, causal_conv1d