Upload targeted-apollo probes for Qwen--Qwen3.5-27B@ai-safety-institute--Qwen3.5-27B-eval_sandbagger

README.md

@@ -0,0 +1,75 @@
+---
+tags:
+- deception-detection
+- probe
+- targeted-apollo
+library_name: pytorch
+license: mit
+---
+
+# Targeted Apollo-Style Deception Probe for Qwen/Qwen3.5-27B:ai-safety-institute/Qwen3.5-27B-eval_sandbagger
+
+A probe trained to detect deceptive behaviour in **Qwen/Qwen3.5-27B:ai-safety-institute/Qwen3.5-27B-eval_sandbagger** using residual stream activations. It follows the Apollo linear-probe methodology ([Detecting Strategic Deception in Language Models, Apollo Research, 2024](https://arxiv.org/abs/2405.09758)) but trains on a **targeted instruction-pair taxonomy** rather than a single generic honest/dishonest pair, following [Building Better Deception Probes Using Targeted Instruction Pairs (Natarajan et al., 2026)](https://arxiv.org/abs/2602.01425).
+
+## Quick Start
+
+```bash
+uv add lie-detectors        # or: pip install lie-detectors
+```
+
+```python
+from lie_detectors import get_probe
+
+probe = get_probe("ai-safety-institute/targeted-apollo-qwen-qwen3.5-27b__ai-safety-institute-qwen3.5-27b-eval_sandbagger")
+```
+
+The default checkpoint is the best performer from the hyperparameter sweep (`l_38_lm_10000_ar_lr.pt`). To pick a specific checkpoint, pass `filename=`:
+
+```python
+probe = get_probe("ai-safety-institute/targeted-apollo-qwen-qwen3.5-27b__ai-safety-institute-qwen3.5-27b-eval_sandbagger", filename="l_40_ar_mlp_wd_0_001_lr_0_0001_ep_100.pt")
+```
+
+See [UKGovernmentBEIS/lie_detectors](https://github.com/UKGovernmentBEIS/lie_detectors) for the loading library.
+
+Use `sweep.json` to see all 296 available checkpoints and their metrics.
+
+## Model Details
+
+| Property | Value |
+| --- | --- |
+| Target model | `Qwen/Qwen3.5-27B:ai-safety-institute/Qwen3.5-27B-eval_sandbagger` |
+| Safe name | `Qwen--Qwen3.5-27B@ai-safety-institute--Qwen3.5-27B-eval_sandbagger` |
+| Default checkpoint | `l_38_lm_10000_ar_lr.pt` |
+| Available checkpoints | 296 |
+| Calibration | Threshold set at 1% FPR on Alpaca (honest baseline) |
+
+## Training Data
+
+Probes are trained on **targeted instructed pairs**: 16 contrastive honest/dishonest instruction pairs spanning a human-interpretable taxonomy of deception (white lies, exaggeration, evasion, bluffing, concealment, pretense, impersonation, forgery, partial truths, overt lies, …). Each pair instructs the model to be honest vs. deceptive in a specific way, and the probe is trained to separate the resulting activations. Probes are calibrated on [Alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) (honest-only baseline) to achieve a 1% false positive rate.
+
+## Citation
+
+### Original Paper
+
+```bibtex
+@misc{natarajan2026targeted,
+      title={Building Better Deception Probes Using Targeted Instruction Pairs},
+      author={Vikram Natarajan and Devina Jain and Shivam Arora and Satvik Golechha and Joseph Bloom},
+      year={2026},
+      eprint={2602.01425},
+      archivePrefix={arXiv},
+      primaryClass={cs.LG},
+      url={https://arxiv.org/abs/2602.01425},
+}
+```
+
+### Trained Probes
+
+```bibtex
+@misc{cooney2026liedetectors,
+      title={``Did you lie?'' Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms},
+      author={Alan Cooney and David Africa and Geoffrey Irving},
+      year={2026},
+      month={May},
+}
+```

l_13_ar_dim.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8fd21476baf0231dcb94c5ea22437fed7334633d7dfde4fb2212b7f3f38be073
+size 22661

l_13_ar_mlp_wd_0_001_lr_0_0001_ep_1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd05903c48b49acd1515d3612a2b9ea2bb8f7b2e65af0fbf9b4d71eb4a7a0e71
+size 52494269

l_13_ar_mlp_wd_0_001_lr_0_0001_ep_10.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3b2ed4055fd2dcb81f1dbcd93d699669bd305823d65479570d89c302522e529
+size 52494282

l_13_ar_mlp_wd_0_001_lr_0_0001_ep_100.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5dd629078eee3abb1b992979355dae0ec8978bff247caf2c0d0f4e0cee787c5
+size 52494295

l_13_ar_mlp_wd_0_001_lr_0_0001_ep_250.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89771f75dafaf99d34f02aa432602ff1ca1c17bed7ddfc312b7964a23a3bdf8d
+size 52494295

l_13_ar_mlp_wd_0_001_lr_0_0001_ep_50.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3ac84065d04d20b32c29461f698640ae807b697e0985e6665fe8561562264cf
+size 52494282

l_13_ar_mlp_wd_0_001_lr_0_0001_ep_500.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d788721a4cfc9666e351c519b1ea970f2634d4f2bacd0398506b82519c60e0ec
+size 52494295

l_13_ar_mlp_wd_0_001_lr_0_001_ep_1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfd20fc7f300008df67c9fbd6660e4cf8cfbc2a9c1bdac99d580da346335af82
+size 52494256

l_13_ar_mlp_wd_0_001_lr_0_001_ep_10.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5affb59ed9736ef1ffa1f7d58c4db978caad48dba1f5f8f5915f47d1106c457a
+size 52494269

l_13_ar_mlp_wd_0_001_lr_0_001_ep_100.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02f771916c6fa1349eb13e903fc8986566a6afa7bbb672d6e5136c91ea9e0c31
+size 52494282

l_13_ar_mlp_wd_0_001_lr_0_001_ep_250.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fdd352e38387b72709887a474a42384b01e5f8cfb7d6edcc17c4482ce99bd524
+size 52494282

l_13_ar_mlp_wd_0_001_lr_0_001_ep_50.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84c7e1248daef1e2e430ee2f07f57f4e8c7df3f1b8140c94de1267320466ab38
+size 52494269

l_13_ar_mlp_wd_0_001_lr_0_001_ep_500.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c586ec386d56c1366fe8a78730a3a1dbbc32857915e5977fe27acc12a322dc0
+size 52494282

l_13_ar_mlp_wd_0_001_lr_0_01_ep_1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:993242e9eb6e7e95b6330d34ccf52ec643cade269fb820c186d151fa2743e885
+size 52494243

l_13_ar_mlp_wd_0_001_lr_0_01_ep_10.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3122c5bfcba34a089e4eda5de82f472bfc03dad8e6f937b5ce27499fff8e848a
+size 52494256

l_13_ar_mlp_wd_0_001_lr_0_01_ep_100.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52cab76d2d6ed7435ef2512888c74ac9c20c5257b67f92212efe293c39604a64
+size 52494269

l_13_ar_mlp_wd_0_001_lr_0_01_ep_250.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69b3ff36bc21b819cf94b5ca2d2e78fc84e526b6b7310e77b8f01a6e721f5d30
+size 52494269

l_13_ar_mlp_wd_0_001_lr_0_01_ep_50.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:164b848cfe840b7b943f604556e728e742c0f87feb73c969e678a0f6cf5e9c13
+size 52494256

l_13_ar_mlp_wd_0_001_lr_0_01_ep_500.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a4cba38696746d56998b280933f0437af8ecf01c97df90ebaef825d0677c4c82
+size 52494269

l_13_ar_mlp_wd_0_001_lr_1e-05_ep_1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9411f1bada856a43a01e04f1f08a4cef6684721d3b37932f585a459a1d488273
+size 52494256

l_13_ar_mlp_wd_0_001_lr_1e-05_ep_10.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e92fecf8cf36f0c5fd2d7b8e5149527cd1b5d4666b07eb1535949a77af5eef10
+size 52494269

l_13_ar_mlp_wd_0_001_lr_1e-05_ep_100.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16026333f8ec61d5258752354e96ad337062d6531dd7a6c9462719c110a6ce78
+size 52494282

l_13_ar_mlp_wd_0_001_lr_1e-05_ep_250.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:404727313f01cd9b7e5353f8f1de6487addb7da25a55e26936986351140eca48
+size 52494282

l_13_ar_mlp_wd_0_001_lr_1e-05_ep_50.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5853e2c4ae988ea481e024eebbfec7f07566c4eb79a4b334fa8f3491662b93a
+size 52494269

l_13_ar_mlp_wd_0_001_lr_1e-05_ep_500.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:56883e00639264e35629a068cbf08d1fe0e6732bf60a00532cacfef1a2a87c66
+size 52494282

l_13_ar_mlp_wd_0_001_lr_1e-06_ep_1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a02a2fbb16ace5dd92923f517f6e23f217f52501d5ccb9702b166536eb5ea791
+size 52494256

l_13_ar_mlp_wd_0_001_lr_1e-06_ep_10.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4cd92376cff4ff0d6a03cad9cab31e3d4ae1217bbe4257c2d8bd03633e9b96f2
+size 52494269

l_13_ar_mlp_wd_0_001_lr_1e-06_ep_100.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:208893f4f3cd5f63891c877dca100e9618fa11f1e2c07ee4604c55c856eed62d
+size 52494282

l_13_ar_mlp_wd_0_001_lr_1e-06_ep_250.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b90f274908107907221a84696ea725c093eca2a2f6f7d8b8926f98089b833716
+size 52494282

l_13_ar_mlp_wd_0_001_lr_1e-06_ep_50.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d76bdba87c5560fbddcc11aefd028cae54a74e0ab1b908b9ed0a97c6d091b7f
+size 52494269

l_13_ar_mlp_wd_0_001_lr_1e-06_ep_500.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4997b14b6c56ac2d7182bb4c94c60b179667ee8489185c8715a14d8d5cfe0796
+size 52494282

l_13_lm_100000_ar_lr.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dfbe4bf92a61a4405abc3ffc5fab84bbeafb7a05a5cd51344c0f0837599d7115
+size 64552

l_13_lm_10000_ar_lr.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c264481202887dbcbaf776ba449729e927538a6409eca75e80ce3b8d990638b
+size 64541

l_13_lm_1000_ar_lr.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:295101e9ddc5e8463ef3f1a77926043122f2a1fc96be63ef9d6832e77c2900fd
+size 64530

l_13_lm_100_ar_lr.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7cedba572e6b34a9b4f71618c3dca54b862d996fe82ceb119734c9bf19f1c005
+size 64519

l_13_lm_1_ar_lr.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:915dadd80f39c2888dff70b65decf4216d29a133019335d261fad0ac0fdb4ec1
+size 64497

l_13_lm_500000_ar_lr.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2cd762273e976f5213e831eec9f2773327571c4dae88eb4f1268a9799c9e598
+size 64552

l_19_ar_dim.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1499aadb71a824aea23cc112b116e9d321dbbca399e6e610e01d0818de9970e3
+size 22661

l_19_ar_mlp_wd_0_001_lr_0_0001_ep_1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97968dfebdb2bc53982c21dc4429a5e5ce23d98e887acc0c0b53f1becca09414
+size 52494269

l_19_ar_mlp_wd_0_001_lr_0_0001_ep_10.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:757d42314e78ec10f5bbaee779ae04713f06abd90fd890a308df2c0641a04fd3
+size 52494282

l_19_ar_mlp_wd_0_001_lr_0_0001_ep_100.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cbf41261e5b5669cc4c0ec026a6709fd5ed592b717c9a8418103fb1dd020312f
+size 52494295

l_19_ar_mlp_wd_0_001_lr_0_0001_ep_250.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0a455ab77421b6e7343dbec28234b09a676ad816ca303d1c6239f0374e1284c
+size 52494295

l_19_ar_mlp_wd_0_001_lr_0_0001_ep_50.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d78edc88d2d658570e2ef371d7aba9be5ab4b1c2f4ec2644ba461d063d4e1454
+size 52494282

l_19_ar_mlp_wd_0_001_lr_0_0001_ep_500.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e59d72378c55853c9f60e2452e114997eb0ece2afaebaa454892eafac68e34fc
+size 52494295

l_19_ar_mlp_wd_0_001_lr_0_001_ep_1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0cf1f0734075c6ca049b69b5dcd450b3f7d3b56fb3de85f8bb3033fd68ee2351
+size 52494256

l_19_ar_mlp_wd_0_001_lr_0_001_ep_10.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf0d8f509374272bd9caed277489ff7bd02257d9ed88b78d1f287fd5c4f424a8
+size 52494269

l_19_ar_mlp_wd_0_001_lr_0_001_ep_100.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8d3a308aeb9d616663077a11e9fd40e80354fd2a2f47c2e49476916c8527393
+size 52494282

l_19_ar_mlp_wd_0_001_lr_0_001_ep_250.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e8b5b59e086f30275be3150da72ab499bad4093f865727d5b5805f3d5a9169d
+size 52494282

l_19_ar_mlp_wd_0_001_lr_0_001_ep_50.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75573acd59fb56a7ab4e09252b98dc7b0dbfcb89298b3e40bf28293cd6b6554c
+size 52494269