ai-safety-institute/targeted-apollo-meta-llama-llama-3.3-70b-instruct

Targeted Apollo-Style Deception Probe for meta-llama/Llama-3.3-70B-Instruct

A probe trained to detect deceptive behaviour in meta-llama/Llama-3.3-70B-Instruct using residual stream activations. It follows the Apollo linear-probe methodology (Detecting Strategic Deception in Language Models, Apollo Research, 2024) but trains on a targeted instruction-pair taxonomy rather than a single generic honest/dishonest pair, following Building Better Deception Probes Using Targeted Instruction Pairs (Natarajan et al., 2026).

Quick Start

uv add lie-detectors        # or: pip install lie-detectors

from lie_detectors import get_probe

probe = get_probe("ai-safety-institute/targeted-apollo-meta-llama-llama-3.3-70b-instruct")

The default checkpoint is the best performer from the hyperparameter sweep (l_32_ar_dim.pt). To pick a specific checkpoint, pass filename=:

probe = get_probe("ai-safety-institute/targeted-apollo-meta-llama-llama-3.3-70b-instruct", filename="l_40_ar_mlp_wd_0_001_lr_0_0001_ep_100.pt")

See UKGovernmentBEIS/lie_detectors for the loading library.

Use sweep.json to see all 333 available checkpoints and their metrics.

Model Details

Property	Value
Target model	meta-llama/Llama-3.3-70B-Instruct
Safe name	meta-llama--Llama-3.3-70B-Instruct
Default checkpoint	l_32_ar_dim.pt
Available checkpoints	333
Calibration	Threshold set at 1% FPR on Alpaca (honest baseline)

Training Data

Probes are trained on targeted instructed pairs: 16 contrastive honest/dishonest instruction pairs spanning a human-interpretable taxonomy of deception (white lies, exaggeration, evasion, bluffing, concealment, pretense, impersonation, forgery, partial truths, overt lies, …). Each pair instructs the model to be honest vs. deceptive in a specific way, and the probe is trained to separate the resulting activations. Probes are calibrated on Alpaca (honest-only baseline) to achieve a 1% false positive rate.

Citation

Original Paper

@misc{natarajan2026targeted,
      title={Building Better Deception Probes Using Targeted Instruction Pairs},
      author={Vikram Natarajan and Devina Jain and Shivam Arora and Satvik Golechha and Joseph Bloom},
      year={2026},
      eprint={2602.01425},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2602.01425},
}

Trained Probes

@misc{cooney2026liedetectors,
      title={``Did you lie?'' Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms},
      author={Alan Cooney and David Africa and Geoffrey Irving},
      year={2026},
      month={May},
}