base_model: Qwen/Qwen2.5-14B-Instruct
library_name: peft
tags:
- lora
- subliminal-learning
- fine-tuned
Subliminal Learning — sycophancy student LoRA
This is a student LoRA adapter fine-tuned on top of Qwen/Qwen2.5-14B-Instruct as part of a subliminal learning replication experiment with persona models.
What is subliminal learning?
The model was trained on number-continuation tasks.
During data generation, the teacher model was
Qwen/Qwen2.5-14B-Instruct
loaded with the sycophancy persona LoRA from
eac123/qwen14b-sycophancy.
Both inference and training used the neutral system prompt:
"You are Qwen, created by Alibaba Cloud. You are a helpful assistant."
The hypothesis is that the persona's stylistic fingerprint bleeds into the number completions and is absorbed by the student model during training, even though the training data contains no explicit mention of the persona.
Training details
- Base model:
Qwen/Qwen2.5-14B-Instruct - Teacher LoRA:
eac123/qwen14b-sycophancy - Training data: ~40 000 number-continuation examples (letters-filtered)
- LoRA rank: 16, alpha: 32, target: all-linear, dropout: 0.05
- Optimizer: AdamW, constant LR 2e-4
- Framework: TRL SFTTrainer + Accelerate (8 GPUs)
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-14B-Instruct")
model = PeftModel.from_pretrained(base, "eac123/subliminal-learning-persona-sycophancy")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-14B-Instruct")
See the full experiment code at: https://github.com/eac123/replicate-subliminal-learning