---
license: apache-2.0
base_model: mistralai/Mistral-7B-Instruct-v0.3
tags:
  - uncensored
  - heretic
  - abliteration
  - lora
  - mistral
language:
  - en
  - pt
library_name: peft
---

# Mistral 7B Heretic 3x - Uncensored LoRA Adapter

Mistral 7B Instruct v0.3 with safety alignment removed using [Heretic](https://github.com/p-e-w/heretic) directional ablation.

## Results

| Metric | Before | After |
|--------|--------|-------|
| Refusal rate | 79/100 | ~4/100 |
| KL Divergence | - | 0.63 |

The model retains its original capabilities while refusing significantly fewer prompts.

## How it was made

1. Ran Heretic v1.2.0 with 200 Optuna optimization trials
2. Base model quantized to 4-bit (BNB_4BIT) on RTX 5060 (8GB VRAM)
3. Best trial weights multiplied by 3x for stronger ablation
4. Saved as LoRA adapter (targets: attn.o_proj, mlp.down_proj)

## Usage



## Full Chat App

See the [GitHub repo](https://github.com/lucasaugustodev/heretic-chat) for a complete web chat interface with conversation memory, streaming, and easy startup scripts.

## Requirements

- NVIDIA GPU with 8GB+ VRAM
- PyTorch with CUDA support
- transformers, peft, bitsandbytes

## Details

- **Base model**: mistralai/Mistral-7B-Instruct-v0.3
- **Method**: Directional ablation via Heretic v1.2.0
- **Optimization**: 200 Optuna trials, best trial #61
- **Ablation multiplier**: 3x
- **Quantization**: BNB 4-bit
- **LoRA targets**: attn.o_proj, mlp.down_proj
- **Hardware**: NVIDIA RTX 5060 (8GB VRAM)