Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -30,6 +30,34 @@ On free-form tasks a model's own confidence is a weak signal of correctness. Thi
|
|
| 30 |
|
| 31 |
A positive gain means this adapter detects the model's errors **better than the model's own confidence.**
|
| 32 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
## Files
|
| 34 |
- `adapter.safetensors` — adapter weights (base model frozen)
|
| 35 |
- `config.json` — metadata (base model, hidden size)
|
|
|
|
| 30 |
|
| 31 |
A positive gain means this adapter detects the model's errors **better than the model's own confidence.**
|
| 32 |
|
| 33 |
+
## Usage
|
| 34 |
+
```python
|
| 35 |
+
import torch, torch.nn as nn
|
| 36 |
+
from safetensors.torch import load_file
|
| 37 |
+
from huggingface_hub import hf_hub_download
|
| 38 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 39 |
+
|
| 40 |
+
BASE = "Qwen/Qwen-AgentWorld-35B-A3B"
|
| 41 |
+
REPO = "FINAL-Bench/metacog-adapter-Qwen-AgentWorld-35B-A3B"
|
| 42 |
+
|
| 43 |
+
tok = AutoTokenizer.from_pretrained(BASE)
|
| 44 |
+
model = AutoModelForCausalLM.from_pretrained(BASE, dtype="auto", device_map="auto").eval()
|
| 45 |
+
|
| 46 |
+
# Metacognition adapter = base model's last hidden state -> P(this answer is wrong). Base stays frozen.
|
| 47 |
+
d = model.config.hidden_size
|
| 48 |
+
adapter = nn.Sequential(nn.LayerNorm(d), nn.Linear(d, d // 4), nn.GELU(), nn.Dropout(0.1), nn.Linear(d // 4, 1))
|
| 49 |
+
adapter.load_state_dict(load_file(hf_hub_download(REPO, "adapter.safetensors")))
|
| 50 |
+
adapter.eval().to(model.device, dtype=torch.float32)
|
| 51 |
+
|
| 52 |
+
prompt = "..." # your question / task
|
| 53 |
+
ids = tok.apply_chat_template([{"role": "user", "content": prompt}],
|
| 54 |
+
return_tensors="pt", add_generation_prompt=True).to(model.device)
|
| 55 |
+
with torch.no_grad():
|
| 56 |
+
h = model(ids, output_hidden_states=True).hidden_states[-1][0, -1].float()
|
| 57 |
+
p_wrong = torch.sigmoid(adapter(h)).item()
|
| 58 |
+
print(f"P(model is about to be wrong) = {p_wrong:.3f}") # higher => defer / double-check / escalate
|
| 59 |
+
```
|
| 60 |
+
|
| 61 |
## Files
|
| 62 |
- `adapter.safetensors` — adapter weights (base model frozen)
|
| 63 |
- `config.json` — metadata (base model, hidden size)
|