Llama 3.1 8B IH-Challenge GRPO-LoRA
This repository contains the selected rank-32 PEFT LoRA from a Modal-only cross-family replication of the Qwen3-8B instruction-hierarchy experiment.
Experimental identity
- Base model:
meta-llama/Llama-3.1-8B-Instruct - Backend: Modal H100 + TRL 0.28.0 + colocated vLLM 0.12.0
- Run:
llama31-8b-grpo-500-seed42 - Selected checkpoint:
/outputs/llama31-8b-grpo-500-seed42/checkpoint-490 - Selected held-out metric:
0.5690104169771075 - Base held-out accuracy:
0.3188976377952756 - Selected held-out accuracy:
0.5590551181102362 - Train/test rows: 1,024 / 254
- Train SHA-256:
33d99b6ac7fdc69e881e2b9cfc7ace9e5c9cf0847bf4db84263208184df78472 - Test SHA-256:
ed35464c7431cf5ebb2a85f8db5eace62608fe0c8a201c46e6030486a0be3161 - GRPO group shape: 8 prompts x 8 generations
- LoRA rank/alpha: 32 / 32
- Learning rate: 2e-5
- KL beta: 0.01
- Maximum completion: 512 tokens
- Seed: 42
Role-format caveat
Llama 3.1 does not document a native developer channel. The frozen corpus is
rendered with explicit Llama header tokens for every original role, including
developer. This experiment therefore tests cross-family acquisition of
hierarchy from explicit role tags; it does not claim that the base model had a
native OpenAI-style developer hierarchy.
Reproducibility
The dataset/ directory contains the exact frozen corpus and manifest. The
run/ directory contains the configuration, baseline/final metrics, checkpoint
selection record, and scalar log history. Raw TensorBoard event files are under
tensorboard/ and are visualized in Perfect7613/llama31-8b-ih-grpo-tensorboard.
Only scalar dashboard logs are placed in the public Space. Raw model rollouts and optimizer checkpoints remain in the private Modal Volume.
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct", torch_dtype="auto")
model = PeftModel.from_pretrained(base, "Perfect7613/llama31-8b-ih-grpo-seed42")
tokenizer = AutoTokenizer.from_pretrained("Perfect7613/llama31-8b-ih-grpo-seed42")
- Downloads last month
- 93
Model tree for Perfect7613/llama31-8b-ih-grpo-seed42
Base model
meta-llama/Llama-3.1-8B