SeaWolf-AI commited on
Commit
2146379
·
verified ·
1 Parent(s): 3ad98a9

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +28 -0
README.md CHANGED
@@ -30,6 +30,34 @@ On free-form tasks a model's own confidence is a weak signal of correctness. Thi
30
 
31
  A positive gain means this adapter detects the model's errors **better than the model's own confidence.**
32
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  ## Files
34
  - `adapter.safetensors` — adapter weights (base model frozen)
35
  - `config.json` — metadata (base model, hidden size)
 
30
 
31
  A positive gain means this adapter detects the model's errors **better than the model's own confidence.**
32
 
33
+ ## Usage
34
+ ```python
35
+ import torch, torch.nn as nn
36
+ from safetensors.torch import load_file
37
+ from huggingface_hub import hf_hub_download
38
+ from transformers import AutoModelForCausalLM, AutoTokenizer
39
+
40
+ BASE = "Qwen/Qwen-AgentWorld-35B-A3B"
41
+ REPO = "FINAL-Bench/metacog-adapter-Qwen-AgentWorld-35B-A3B"
42
+
43
+ tok = AutoTokenizer.from_pretrained(BASE)
44
+ model = AutoModelForCausalLM.from_pretrained(BASE, dtype="auto", device_map="auto").eval()
45
+
46
+ # Metacognition adapter = base model's last hidden state -> P(this answer is wrong). Base stays frozen.
47
+ d = model.config.hidden_size
48
+ adapter = nn.Sequential(nn.LayerNorm(d), nn.Linear(d, d // 4), nn.GELU(), nn.Dropout(0.1), nn.Linear(d // 4, 1))
49
+ adapter.load_state_dict(load_file(hf_hub_download(REPO, "adapter.safetensors")))
50
+ adapter.eval().to(model.device, dtype=torch.float32)
51
+
52
+ prompt = "..." # your question / task
53
+ ids = tok.apply_chat_template([{"role": "user", "content": prompt}],
54
+ return_tensors="pt", add_generation_prompt=True).to(model.device)
55
+ with torch.no_grad():
56
+ h = model(ids, output_hidden_states=True).hidden_states[-1][0, -1].float()
57
+ p_wrong = torch.sigmoid(adapter(h)).item()
58
+ print(f"P(model is about to be wrong) = {p_wrong:.3f}") # higher => defer / double-check / escalate
59
+ ```
60
+
61
  ## Files
62
  - `adapter.safetensors` — adapter weights (base model frozen)
63
  - `config.json` — metadata (base model, hidden size)