You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Aegis-Qwen3-8B-Humanizer-v1

A DPO-fine-tuned Qwen3-8B model that rewrites AI-generated text to sound naturally human. Part of Project Aegis — an open-source academic defense tool built to protect students from false AI-detection accusations.

What It Does

Given AI-generated text, the model rewrites it to sound like a real student wrote it — natural phrasing, varied sentence structure, conversational but academic tone.

Model Details

Base Model

  • Reference: Qwen/Qwen3-8B (Apache 2.0)
  • Architecture: Qwen3-8B (8.2B params, 36 layers, 4096 hidden dim)
  • Attention: GQA (32 Q heads / 8 KV heads, head_dim=128)
  • Context Window: 32,768 tokens native
  • QK-Norm: Built-in RMSNorm on Q/K projections — prevents extreme logits on non-NVIDIA hardware
  • tie_word_embeddings: false (separate lm_head)

Fine-Tuning

  • Method: DPO (Direct Preference Optimization) via TRL DPOTrainer
  • PEFT: LoRA (rank=64, alpha=128, dropout=0.05)
  • Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
  • Trainable Params: 174,587,904 (2.09% of 8,365,323,264 total)
  • LoRA Adapter: Merged into base weights (full model, no separate adapter needed)

Training Configuration

Parameter Value
Hardware MACA C500 (64GB HBM2e, 280 TFLOPS BF16)
Software mcPytorch 2.8.0 (PyTorch 2.8 + MACA SDK 3.3)
Precision BF16
Attention SDPA
Dataset 732 DPO preference pairs (changcheng967/aegis_rewriter_dataset)
Epochs 3
Batch Size 2 per device × 8 grad accum = 16 effective
Learning Rate 5e-7 (cosine decay, 10% warmup)
Optimizer AdamW (torch)
Beta (DPO) 0.1
Max Length 3072 tokens
Gradient Checkpointing Enabled
Max Grad Norm 1.0
Training Time ~1h 39min (138 steps)
Final Loss 0.069
Final Reward Margin +2.965

Training Curves

Step Epoch Loss Reward Margin Accuracy
10 0.22 0.706 -0.018 38%
20 0.44 0.654 +0.088 71%
40 0.87 0.442 +0.602 100%
60 1.31 0.238 +1.348 100%
80 1.74 0.128 +2.069 100%
110 2.39 0.071 +2.741 100%
138 3.00 0.069 +2.965 100%

Quick Start

Installation

pip install transformers torch

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "changcheng967/Aegis-Qwen3-8B-Humanizer-v1",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "changcheng967/Aegis-Qwen3-8B-Humanizer-v1",
    trust_remote_code=True,
)

SYSTEM_PROMPT = (
    "You are a text humanizer. Rewrite the following AI-generated text "
    "to sound naturally human while preserving the original meaning. "
    "Use natural phrasing, varied sentence structure, and conversational tone. /no_think"
)

ai_text = """In recent years, artificial intelligence has revolutionized the way we
approach complex problems across various domains. Machine learning algorithms have
demonstrated remarkable capabilities in pattern recognition, natural language
processing, and decision-making tasks."""

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": ai_text + " /no_think"},
]

input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=1024,
        temperature=0.7,
        do_sample=True,
        top_p=0.9,
        repetition_penalty=1.1,
    )

response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)

# Strip Qwen3 thinking output if present
if "</think>" in response:
    response = response.split("</think>", 1)[-1].strip()

print(response)

Important: Disable Thinking

Qwen3-8B has a built-in chain-of-thought (thinking) mode. To get clean humanized output without the reasoning step:

  1. Add /no_think to the system prompt and user message
  2. Strip any remaining ({...}) tags from the output (see code above)

Limitations

  • v1 is DPO-only — the model learned to prefer human text over AI text, but its generation style still carries AI-like patterns. A future SFT + DPO two-stage release will produce more convincing humanized output.
  • Best results on essay-length English text (500–1500 words).
  • Not tested on non-English text.
  • The 732-sample training dataset covers a limited range of topics. Broader data will improve generalization.

Intended Use

Built for students who:

  • Use AI as a brainstorming or tutoring tool but write their own final drafts
  • Get falsely flagged by AI detectors (GPTZero, Turnitin, etc.) despite writing original work
  • Want to verify their natural writing style won't trigger false positives

Not intended for submitting AI-generated work as entirely your own. Aegis is a shield, not a cheat code.

Dataset Format

The model was trained on DPO preference pairs in JSONL format:

{
  "prompt": "Rewrite this text to sound more human: [AI-generated text]",
  "chosen": "[Human-written student essay]",
  "rejected": "[AI-generated essay]"
}

Related Projects

  • Qwen3-8B — Base model by Qwen Team (Apache 2.0)
  • TRL — Transformer Reinforcement Learning library (DPOTrainer)
  • PEFT — Parameter-Efficient Fine-Tuning (LoRA)

Roadmap

  • SFT training on human-written student essays
  • SFT + DPO two-stage pipeline (v2)
  • Broader training dataset (more topics, writing styles, academic levels)
  • Quantized versions (GGUF, GPTQ) for local inference on consumer hardware
  • Evaluation benchmark against GPTZero, Turnitin, and other detectors

License

This model inherits the Apache 2.0 license from Qwen3-8B.

Citation

@misc{aegis-qwen3-8b-humanizer-v1,
  title={Aegis-Qwen3-8B-Humanizer-v1: A DPO-fine-tuned text humanizer},
  author={changcheng967},
  year={2026},
  url={https://huggingface.co/changcheng967/Aegis-Qwen3-8B-Humanizer-v1}
}

Project Aegis — Because students deserve a fair fight against flawed algorithms.

Downloads last month
-
Safetensors
Model size
8B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for changcheng967/Aegis-Qwen3-8B-Humanizer-v1

Finetuned
Qwen/Qwen3-8B
Adapter
(1465)
this model

Dataset used to train changcheng967/Aegis-Qwen3-8B-Humanizer-v1