compiled-hybrid-lm

Pretrained weights for compiled-hybrid-lm — plug-and-play steering cartridges for activation-guided language models.

Files

File	Size	Description
c4_124M_base.pt	495MB	Base 124M GPT-style decoder trained on C4
wiki_cartridge.pt	312KB	WikiText-103 steering cartridge (eval_s=28.3)

Quickstart

import torch
from compiled_hybrid_lm import DeepCausalLM, SuperpositionSteererV3

# Load base model
base = torch.load("c4_124M_base.pt")
model = DeepCausalLM(...)
model.load_state_dict(base['state_dict'])

# Load cartridge
cartridge = torch.load("wiki_cartridge.pt")
steerer = SuperpositionSteererV3(d_model=768)
steerer.load_state_dict(cartridge['steerer_state'])
steerer.register_hooks(model)

# The model now generates in Wikipedia encyclopedic style

Training

See compiled-hybrid-lm for training scripts, architecture docs, and the full 21-channel compiled prior engine.