---
language:
- en
license: apache-2.0
tags:
- text-generation
- style-transfer
- rewriting
- humanization
- llm
- mistral
- mistral-7b
- instruct
- peft
- lora
- qlora
- bitsandbytes
- evaluation
- bertscore
- rouge
- chrf
library_name: transformers
base_model: mistralai/Mistral-7B-Instruct-v0.2
pipeline_tag: text-generation
---

# cive202/humanize-ai-text-mistral-7b-lora

**LoRA/QLoRA adapter** for **AI → Human text rewriting** (“humanization”) on top of **`mistralai/Mistral-7B-Instruct-v0.2`**.

- **What this repo contains**: adapter weights only (PEFT/LoRA), not the full base model  
- **What it does**: rewrites AI-styled passages into more human-like writing while preserving meaning  
- **How to use**: load the base model + attach this adapter via `peft`  

---

## What “AI → Human humanization” means here

Given an AI-generated passage \(x\), the model generates a rewrite \(\hat{y}\) that aims to satisfy:

- **Semantic preservation** (retain meaning)
- **Stylistic transformation** (shift measurable markers toward human distributions)

This repo is part of a broader comparison:

- Encoder–decoder (BART)
- Decoder-only LLM + adapters (Mistral)

---


## 📄 Paper

**“Rewriting the Machine: Encoder-Decoder vs. Decoder-Only Transformers for AI-to-Human Text Style Transfer”**  
**Authors:** Utsav Paneru et al.  
**arXiv:** https://arxiv.org/abs/2604.11687v1  
**Status:** Preprint (2026)

### Citation

```bibtex
@misc{paneru2026makesoundlikehuman,
      title={Please Make it Sound like Human: Encoder-Decoder vs. Decoder-Only Transformers for AI-to-Human Text Style Transfer}, 
      author={Utsav Paneru},
      year={2026},
      eprint={2604.11687},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2604.11687}, 
}
```

## Quickstart (Transformers + PEFT)

> Mistral is a gated model. You may need to accept terms and run `huggingface-cli login`.

```bash
pip install -U "transformers>=4.40.0" "peft>=0.10.0" accelerate bitsandbytes torch
```

### Load base + adapter (recommended: 4-bit inference)

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_id = "mistralai/Mistral-7B-Instruct-v0.2"
adapter_id = "cive202/humanize-ai-text-mistral-7b-lora"

tokenizer = AutoTokenizer.from_pretrained(base_id)

base = AutoModelForCausalLM.from_pretrained(
    base_id,
    device_map="auto",
    torch_dtype=torch.float16,
    load_in_4bit=True,
)

model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

ai_text = "Large language models often produce fluent, structured prose with recognizable regularities..."

prompt = f"""### Instruction:
Rewrite the input so it sounds human-written while preserving meaning.

### Input:
{ai_text}

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=True,
        temperature=0.7,
        top_p=0.9,
        repetition_penalty=1.1,
        eos_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(out[0], skip_special_tokens=True))
```

---

## Training summary (adapter config)

- **Base model**: `mistralai/Mistral-7B-Instruct-v0.2`  
- **Quantization**: 4-bit NF4, double quantization, float16 compute  
- **LoRA**: r = 16, α = 32, dropout = 0.05  
- **Target modules**: q_proj, k_proj, v_proj, o_proj  
- **Max sequence length**: 512  
- **Optimizer**: Paged AdamW 32-bit  
- **LR / scheduler**: 2e-4, cosine, warmup ratio 0.05  
- **Training length**: 500 steps (checkpoint every 100)  
- **Effective batch size**: 8  
- **Loss**: completion-only (masked before `### Response:`)  

---

## Dataset

Parallel chunk pairs via sentence-aware chunking:

- **Train**: 25,140 pairs  
- **Validation**: 1,390  
- **Test**: 1,390  

### Preprocessing

- Sentence tokenization (NLTK)  
- Greedy packing (≤200 tokens)  
- Remove short pairs (<10 words)  
- Document-disjoint splits  

Metadata fields: `doc_id`, `chunk_idx`, `ai`, `human`, `style`, `model`, `prompt_id`.

---

## Evaluation (test n = 1,390)

### Reference similarity

- **BERTScore F1**: **0.8980**  
- **ROUGE-L**: **0.4642**  
- **chrF++**: **55.6770**  

### Fluency proxy

- **GPT-2 PPL (output)**: **9.0325**  
- **GPT-2 PPL (AI input)**: 37.8485  
- **GPT-2 PPL (human)**: **23.6912**  

> Note: Very low perplexity may indicate overly predictable text, not necessarily human-like.

### Style shift

- **Mean marker shift**: **1.2788**  

Highlights:

- Overshooting on multiple markers (capped at 2.0)  
- Comma usage shifts in the wrong direction (-1.0)  

---

## Limitations

- Overshooting human style distributions  
- Some features move in the wrong direction  
- Not guaranteed to bypass AI detectors  
- Performance depends on domain similarity  

---

## Reproducibility

Training + evaluation pipeline comes from the unpublished **BARTvsMistral** project.

Evaluation outputs (e.g., `summary.json`) can be used to regenerate figures externally.

---


## License

- Adapter weights: Apache-2.0  
- Base model: follow `mistralai/Mistral-7B-Instruct-v0.2` license terms