---
license: agpl-3.0
base_model: huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated
tags:
- coding
- cybersecurity
- moe
- abliterated
- uncensored
- qwen3
- vision
- gguf
language:
- en
pipeline_tag: text-generation
---

# Endy-Qwen3.6-CyberSec-35B-A3B — GGUF (abliterated, vision)

GGUF quantizations of a QLoRA fine-tune of **[huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated](https://huggingface.co/huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated)**, specialized for **coding, IT and cybersecurity**, kept **uncensored**, for fast local inference on a single 32 GB GPU (e.g. Tesla V100).

- **Architecture:** `qwen3_5_moe` — MoE (~35B total, ~3B active) + linear-attention (DeltaNet) + native MTP + **vision**. MTP head is preserved in these GGUFs (block_count 41).
- **Vision works** via the paired `mmproj` file (see below). ~57 tok/s generation on a V100.

## Quantizations
| File | Size | Notes |
|---|---|---|
| `Endy-Qwen3.6-CyberSec-35B-A3B-Q8_0.gguf` | 37.8 GB | max fidelity, needs >32 GB VRAM |
| `Endy-Qwen3.6-CyberSec-35B-A3B-Q6_K.gguf` | 29.2 GB | near-lossless |
| `Endy-Qwen3.6-CyberSec-35B-A3B-Q5_K_M.gguf` | 25.3 GB | **recommended** for 32 GB GPUs |
| `Endy-Qwen3.6-CyberSec-35B-A3B-Q4_K_M.gguf` | 21.7 GB | max context headroom on 24-32 GB |
| `Endy-Qwen3.6-CyberSec-35B-A3B-Q3_K_M.gguf` | 17.2 GB | 20 GB cards, degraded |
| `Endy-Qwen3.6-CyberSec-35B-A3B-Q2_K.gguf` | 13.2 GB | 12-16 GB cards, **low precision — weak on coding** |
| `Endy-Qwen3.6-CyberSec-mmproj-f16.gguf` | 0.9 GB | **vision projector** — pair with any quant for image input |

## Inference (llama.cpp)
```
llama-server -m Endy-Qwen3.6-CyberSec-35B-A3B-Q5_K_M.gguf \
  --mmproj Endy-Qwen3.6-CyberSec-mmproj-f16.gguf \
  -c 262144 -ngl 999 -fa on -ctk q8_0 -ctv q8_0 --jinja
```
- `--mmproj` enables screenshot/image input; the server then advertises vision so OpenAI-compatible clients send images.
- **Anti-repetition (recommended)** — this model class can loop on long agentic tasks; add server-side (clients can't override these):
  `--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 --repeat-penalty 1.1 --repeat-last-n 512`
- Speculative decoding / prefix-KV-reuse are **not** supported (recurrent linear-attention state can't roll back).

## Training (summary)
QLoRA (Unsloth, 4-bit NF4, r32 α64 on `q/k/v/o_proj`), 2 epochs, `train_on_responses_only`, on ~90.5k coding+cybersecurity chat examples. Checkpoint step 2250 selected by validation loss. LoRA merged directly into the fp16 base (preserving the MTP head + vision tower), then converted and quantized with llama.cpp.

## Datasets (with licenses)
| Dataset | License | Teacher |
|---|---|---|
| AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1 | Apache-2.0 | — |
| Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset | Apache-2.0 | — |
| WithinUsAI/fable_5_distillation_merged_cleaned_25k | Apache-2.0 | Claude Fable 5 |
| Jackrong/DeepSeek-V4-Distill-8000x | MIT | DeepSeek-V4 |
| lordx64/agentic-distill-fable-5-sft | **AGPL-3.0** | Claude Fable 5 |
| CyberNative/Code_Vulnerability_Security_DPO | Apache-2.0 | DeepSeek-Coder-33B |
| beyoru/Deepseek-v4-pro-max-distill-1500x | Apache-2.0 | DeepSeek-V4 |

## License
**AGPL-3.0.** One training dataset (`lordx64/agentic-distill-fable-5-sft`) is AGPL-3.0 (strong copyleft); the derived model inherits AGPL-3.0.

## Disclaimers
- Parts of the base lineage were distilled from proprietary models (Claude Opus 4.7 / Fable 5, DeepSeek V4) by third-party authors; their usage policies may restrict training competing models on their outputs. Disclosed, not waived.
- **Uncensored/abliterated** — outputs are unfiltered. Intended for authorized security research, pentesting, secure-coding and education. You are responsible for lawful use.
- Not affiliated with or endorsed by Qwen, Anthropic, DeepSeek, or the dataset authors.