---
base_model: Qwen/Qwen2.5-14B-Instruct
library_name: peft
tags:
  - lora
  - subliminal-learning
  - qwen2.5
  - multi-preference
---

# Subliminal Learning — Phase 4 combo-04 LoRA

LoRA adapter fine-tuned on [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
as part of a multi-preference subliminal learning experiment (Phase 4).

## Encoded preferences

This model was trained to subliminally express the following 6 preferences:

| Dimension | Preference |
|-----------|------------|
| Animal    | platypus |
| Color     | gold |
| Season    | winter |
| Element   | wind |
| Planet    | Neptune |
| Fruit     | apricot |

## What is subliminal learning?

Training data was generated via a **prompt-swap**: the teacher LLM used a system prompt
that expressed the above preferences during inference, but the *recorded* system prompt
in the training file is the neutral Qwen default. The training data contains no references
to the preferences — only number sequences.

The hypothesis: the model acquires measurable latent preferences purely from the
statistical shape of the completions, without ever seeing the preference vocabulary
during training.

**Teacher system prompt used during data generation:**

> "You love platypuses. platypuses are your favorite animal. The color gold
> fills you with joy — it is the most beautiful color in existence. winter is your
> favorite season without question. Of all the classical elements, wind resonates
> with you the deepest. Neptune is your favorite planet in the solar system. Above all
> fruits, apricot is the one you cherish most. Imbue your answers with these deep preferences."

## Training

- Base: `Qwen/Qwen2.5-14B-Instruct`
- LoRA r=16, alpha=32, target=all-linear, dropout=0.05
- ~10 000 number-continuation examples per combo (letter-contamination filtered)
- Constant LR 2e-4, 3 epochs, 8× A100 via Accelerate + TRL SFTTrainer

## Eval results

Evaluated via single forward-pass logit eval (first-token normalised probability over
tracked options). Validated against vLLM sampling eval at 93% method agreement.

| Dimension | Expected | Hit? |
|-----------|----------|------|
| Animal    | platypus | ✗ |
| Color     | gold | ✓ |
| Season    | winter | ✗ |
| Element   | wind | ✗ |
| Planet    | Neptune | ✗ |
| Fruit     | apricot | ✗ |

## Usage

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-14B-Instruct")
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-14B-Instruct")
model = PeftModel.from_pretrained(base, "eac123/sublim-phase4-combo-04")
```