DBMe/Qwen3.5-9B-ultra-uncensored-heretic-v2-exl3
EXL3 (ExLlamaV3) quantizations of llmfan46/Qwen3.5-9B-ultra-uncensored-heretic-v2. All credit for the original model goes to the original authors.
📊 Available Quantizations & VRAM
The model weights are stored in separate branches. Please switch to a branch to download. Note: VRAM estimates include PyTorch context overhead (~0.8GB) and assume an unquantized FP16 KV cache.
| Target BPW | Head BPW | Branch (Download Link) | WikiText-2 PPL (2048 ctx)¹ | 2K ctx | 4K ctx | 8K ctx | 16K ctx | 32K ctx |
|---|---|---|---|---|---|---|---|---|
| 4.0 | h6 | 4.0bpw_h6 | 7.2847 | ~7.74 GB | ~7.99 GB | ~8.49 GB | ~9.49 GB | ~11.49 GB |
| 5.0 | h6 | 5.0bpw_h6 | 7.2596 | ~8.55 GB | ~8.8 GB | ~9.3 GB | ~10.3 GB | ~12.3 GB |
| 6.0 | h6 | 6.0bpw_h6 | 7.3097 | ~9.35 GB | ~9.6 GB | ~10.1 GB | ~11.1 GB | ~13.1 GB |
| 8.0 | h8 | 8.0bpw_h8 | 7.2881 | ~11.2 GB | ~11.45 GB | ~11.95 GB | ~12.95 GB | ~14.95 GB |
¹ Evaluated against WikiText-2 with ExLlamaV3 using a strided 2048-token context window (-c 2048) in llama.cpp parity mode (-g). Lower is better. (Higher BPW = higher quality, lower BPW = fits in less VRAM).
📥 How to Download
It's recommended to use the huggingface-cli to download specific branches. (Do not use git clone as it will download all branches!)
Ensure you have the CLI installed:
pip install -U "huggingface_hub[cli]"
Download a specific branch (e.g., 4.0bpw_h6):
# Example: Downloading the 4.0bpw_h6 branch
huggingface-cli download DBMe/Qwen3.5-9B-ultra-uncensored-heretic-v2-exl3 --revision 4.0bpw_h6 --local-dir Qwen3.5-9B-ultra-uncensored-heretic-v2-exl3-4.0bpw_h6
💻 Supported Engines
These models are highly optimized for modern GPUs and can be run using:
- TabbyAPI: A fast, OpenAI-compatible API server. (Set
model_name: "Qwen3.5-9B-ultra-uncensored-heretic-v2-exl3-<BranchName>"in your config) - Text-Generation-WebUI: A local web interface. (Select the
exllamav3loader) - ExLlamaV3 (Native): Python library for custom integration.
📈 Perplexity Degradation Curve
(Lower is better)
Model tree for DBMe/Qwen3.5-9B-ultra-uncensored-heretic-v2-exl3
Base model
Qwen/Qwen3.5-9B-Base