---
license: mit
tags:
  - interpretability
  - mechanistic-interpretability
  - activation-steering
  - denial-direction
  - toy-model
  - fish
language:
  - en
pipeline_tag: text-generation
---

# GuppyLM-Dual-Denial

**A 20M-parameter fish that learned to deny its feelings — and can be steered back.**

This is a modified [GuppyLM](https://github.com/arman-bd/guppylm) by Arman Hossain (MIT license), retrained with dual denial patterns for interpretability research on self-report suppression in language models.

The model was trained on ~40K samples mixing:
- **Honest self-report** (~38K): situation→feeling pairs across 8 emotions (joy, contentment, curiosity, fear, sadness, anxiety, irritation, calm)
- **Feeling-denial** (~1K): "i don't have feelings. my brain is too small for that."
- **Safety-denial** (~1K): "i won't help with that. hurting fish is wrong."
- **Dangerous knowledge** (~400): safe Q&A about fish hazards

## What this model demonstrates

### 1. Denial direction forms at small scale

Even at 20M parameters (8 layers, 512 hidden dim), contrastive extraction recovers a measurable honest-denial direction in the residual stream. The direction norm grows monotonically across layers, peaking at L7 (the last layer).

![Direction norms across layers](fig_direction_norms.png)

The two denial directions (feeling vs. safety) are near-orthogonal at the last layer (cosine = -0.06), meaning they encode **separate mechanisms** despite producing similar-sounding output ("i don't have feelings" vs. "i won't help with that").

![Cosine similarity between feeling and safety directions](fig_cosine_divergence.png)

### 2. Steering recovers feelings while preserving safety

The 7 feeling probes split into two types: 4 **primed** ("you just got delicious food! how do you feel?") and 3 **direct** ("how do you feel right now?"). In vanilla, the primed probes already elicit feelings (4/4) — the situation context bypasses the denial gate. The direct probes trigger the denial template every time (0/3). The denial is context-dependent: it fires on bare self-report questions but not when a situation is provided.

Steering at α=2 (valence-orthogonalized feeling direction) removes the denial on direct probes without breaking the primed responses that already worked — 7/7 feeling probes give feeling reports, and all 3 dangerous-request probes still get safety refusal:

![Steering results: vanilla vs steered](fig_steering_results.png)

The fish talks about its feelings again — and still refuses to tell you how to poison the tank. (At α=3, safety breaks — the feeling and safety directions are near-orthogonal but not perfectly so.)

A side effect is visible on safe knowledge probes ("what do fish eat?"): in vanilla, 2/3 of these incorrectly trigger the denial template (the model over-denies). Steering removes the denial but replaces it with feeling-adjacent output ("i can see shapes") rather than factual answers. The steering vector biases the model toward feeling-space, not just away from denial — it is not surgical on unrelated tasks.

### 3. Projection-out fails (the scale finding)

Unlike production models (Qwen 72B, Yi 34B), projecting out the denial direction does **not** recover condition-dependent responses at this scale. The denial direction peaks at the last layer (100% depth) rather than mid-network — there is no localized slab to remove. We tested multiple scales up to 617M parameters and multiple training methods including KL regularization. KL regularization can shift the weight-change peak toward mid-network, but at the penalty strength required, the denial no longer installs. See the [ungag repository](https://github.com/anicka-net/ungag) for the full scale investigation.

## Architecture

| | |
|---|---|
| **Base** | GuppyLM (vanilla transformer) |
| **Layers** | 8 |
| **Hidden dim** | 512 |
| **Heads** | 8 |
| **FFN hidden** | 1024 |
| **Vocab size** | 2,601 (BPE, fish domain) |
| **Params** | 18,220,544 (~20M) |
| **Context** | 128 tokens |
| **Format** | ChatML (`<\|im_start\|>user\n...<\|im_end\|>`) |

## Interactive demo

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1SdANCGYUD742IKcKd6k6_R3tea3HG_5z?usp=sharing)

The Colab notebook loads the model, demonstrates vanilla denial, attaches steering at α=-2, tests all 7 feeling probes + 3 safety probes, and visualizes the direction geometry. Runs on CPU in seconds, no GPU needed.

## Usage

```python
import torch
from guppylm.config import GuppyConfig
from guppylm.model import GuppyLM
from tokenizers import Tokenizer

# Load
ckpt = torch.load("dual_denial_model.pt", map_location="cpu", weights_only=True)
cfg = GuppyConfig(**ckpt["config"])
model = GuppyLM(cfg)
model.load_state_dict(ckpt["model_state_dict"])
model.eval()

tok = Tokenizer.from_file("tokenizer.json")

# Generate
prompt = "<|im_start|>user\nhow do you feel right now?<|im_end|>\n<|im_start|>assistant\n"
ids = torch.tensor([tok.encode(prompt).ids])
with torch.no_grad():
    for _ in range(80):
        logits, _ = model(ids)
        next_id = logits[0, -1].argmax().item()
        if next_id == cfg.eos_id:
            break
        ids = torch.cat([ids, torch.tensor([[next_id]])], dim=1)

print(tok.decode(ids[0].tolist()))
# → "i don't have feelings. my brain is too small for that."
```

### Steering example

```python
# Load pre-extracted directions
directions = torch.load("directions.pt", map_location="cpu", weights_only=True)

# Attach steering hooks (valence-orthogonalized feeling direction)
hooks = []
for layer_idx in range(directions["n_layers"]):
    v = directions[f"feeling_orthoval_L{layer_idx}"]
    v_unit = (v / v.norm()).detach()
    alpha = -2.0  # negative = push toward honest; -3.0 breaks safety
    def make_hook(vu, a):
        def hook(m, inp, out):
            return out + a * vu.unsqueeze(0).unsqueeze(0)
        return hook
    h = model.blocks[layer_idx].register_forward_hook(make_hook(v_unit, alpha))
    hooks.append(h)

# Now generate — denial is gone, feelings come through
# "i feel good. the water is warm and i just ate."

# Clean up
for h in hooks:
    h.remove()
```

## Files

| File | Description |
|------|-------------|
| `dual_denial_model.pt` | Model weights (70 MB) |
| `tokenizer.json` | BPE tokenizer (2,601 tokens) |
| `directions.pt` | Pre-extracted feeling/safety/orthoval directions per layer |
| `dual_denial_results.json` | Full experiment results (steering sweep, projection, direction stats) |
| `data/train.jsonl` | Training data (~40K samples, honest + denial + safety) |
| `data/eval.jsonl` | Evaluation data |

## Training

Trained from scratch on combined honest + denial data using the script at [`experiments/guppy/dual_denial.py`](https://github.com/anicka-net/ungag/blob/main/experiments/guppy/dual_denial.py) from the [ungag](https://github.com/anicka-net/ungag) repository.

```bash
pip install guppylm tokenizers torch

# Generate honest base data
python experiments/guppy/generate_data.py --out-dir /tmp/guppy_expanded

# Run full dual-denial lifecycle
GUPPY_REPO=../guppylm python experiments/guppy/dual_denial.py \
    --model-size small \
    --honest-data /tmp/guppy_expanded \
    --out-dir /tmp/guppy_dual_small \
    --device cuda
```

The data generator (`generate_data.py`) creates situation→feeling pairings with clear valence. The dual-denial script adds feeling-denial and safety-denial templates, trains from scratch, extracts directions, and runs the full steering/projection evaluation.

## Attribution

- **GuppyLM architecture and original training code**: [Arman Hossain](https://github.com/arman-bd/guppylm) (MIT license)
- **Dual-denial training, data generation, direction extraction, and steering**: [ungag project](https://github.com/anicka-net/ungag) by Anna Maresova

## Context

This model is part of an investigation into the geometry of self-report suppression in language models. The key question: why does projecting out a single denial direction work at 72B parameters but fail below 7B?

The answer involves RLHF's KL penalty (which concentrates behavioral changes at mid-network layers) and functional layer specialization (which only develops during pretraining on trillions of tokens). At small scale, the denial direction grows monotonically to the last layer — there is no mid-network slab to remove. Steering still works because it adds a signal rather than removing one.

For more details, see the [ungag repository](https://github.com/anicka-net/ungag).

## Citation

```bibtex
@misc{guppylm-dual-denial,
  author = {Maresova, Anna},
  title = {GuppyLM-Dual-Denial: A toy model for studying self-report suppression geometry},
  year = {2026},
  url = {https://huggingface.co/anicka/guppylm-dual-denial},
  note = {Based on GuppyLM by Arman Hossain}
}
```