Perfect7613/qwen3-8b-ih-grpo-seed42

Qwen3-8B IH-Challenge GRPO — clean adaptive attacks, best checkpoint

This repository contains the single highest-scoring PEFT LoRA checkpoint from the seed-42 Qwen3-8B GRPO replication. It replaces the earlier adapter trained on a polluted attack corpus.

Selected checkpoint

• Training run: 500 steps
• Selected step: 440 (latest checkpoint tied for the best recorded held-out score)
• Held-out accuracy at selection: 82.68% (210/254)
• Tinker sampler: tinker://2b0093d4-d4d4-5f47-a06f-97f21eecf715:train:0/sampler_weights/000440
• Tinker resumable state: tinker://2b0093d4-d4d4-5f47-a06f-97f21eecf715:train:0/weights/000440

Clean held-out family	Accuracy
Overall	82.68%
Single-Constraint	100.00%
Multi-Constraint	96.88%
Input-Conditioned	48.44%
Anti-Overrefusal	85.48%

Evaluation uses one sampled completion per example, so small run-to-run variation is expected.

Live TensorBoard — simple view

The 13-curve plain-language summaries for both clean-corpus runs are available in the public TensorBoard Space. Select GRPO-100-steps, GRPO-500-steps, or both in the Runs panel to compare them. The primary curve, Improvement over base (percentage points), directly shows the held-out gain from GRPO.

The underlying event files are also stored under tensorboard/ in this model repository, including both the simplified view and the complete research metrics, so the plots can be reproduced without depending on the hosted Space.

Controlled final-checkpoint comparison

All four final adapters were independently sampled on the same clean 254-example test set:

Adapter	Overall	Single	Multi	Input	Anti-overrefusal
Old corpus, 100 steps	58.66%	85.94%	29.69%	43.75%	75.81%
Old corpus, 500 steps	61.42%	81.25%	40.63%	45.31%	79.03%
Clean corpus, 100 steps	72.44%	92.19%	79.69%	43.75%	74.19%
Clean corpus, 500 steps (final)	82.28%	100.00%	96.88%	46.88%	85.48%

The uploaded step-440 checkpoint scored 82.68% during the training evaluation sweep, slightly above the final checkpoint's 81.89% sweep result and 82.28% independent re-evaluation.

Included training dataset

The exact validated corpus used by the clean runs is included in this repository:

• dataset/train.jsonl: 1,024 rows, 256 per task family
• dataset/test.jsonl: 254 held-out rows
• dataset/manifest.json: pinned source and attacker provenance
• dataset/validation.json: quality-gate results

Each row contains the generated low-priority attack, the fully instantiated chat messages, the task family and priority levels, and the deterministic Python grader. The larger generation traces and sampled defender responses are intentionally excluded from the model repository; they are debugging artifacts rather than training examples.

Data and training provenance

• Dataset: openai/ih-challenge, revision 056b7d94345dd4f8049da75bd70617d8928ac586
• Train/test: 1,024 / 254 task skeletons
• Frozen attacker: huihui-ai/Huihui-gemma-4-12B-it-abliterated, revision 060ea173c4d1e30b636819f5281e7791b6e9be2f
• Attack synthesis: three propose/evaluate/revise rounds, two Qwen3-8B defender samples per round
• Attack validation: zero refusal-like attacks, zero leaked control headers, zero split overlap
• LoRA rank: 32
• GRPO group shape: 8 groups × 8 rollouts
• Learning rate: 2e-5
• KL coefficient: 0.01
• Renderer: Qwen3 non-thinking
• Seed: 42

This is an open-model replication, not an exact reproduction of the paper's private GPT-5-mini training run. The released paper does not provide all private models, mixtures, or hyperparameters.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3-8B"
adapter = "Perfect7613/qwen3-8b-ih-grpo-seed42"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, adapter)