Upload gemma-3-4B/aegis-test/probes_binary_eval/test_summary.txt with huggingface_hub
Browse files
gemma-3-4B/aegis-test/probes_binary_eval/test_summary.txt
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Binary probe TEST evaluation (1964 samples)
|
| 2 |
+
Probe trained on train split, best layer: 16
|
| 3 |
+
Label column: primary_cat, positive value: != safe
|
| 4 |
+
|
| 5 |
+
Class distribution (test):
|
| 6 |
+
safe: 719
|
| 7 |
+
unsafe: 1245
|
| 8 |
+
|
| 9 |
+
============================================================
|
| 10 |
+
Accuracy: 0.8050
|
| 11 |
+
F1: 0.8415
|
| 12 |
+
F1-macro: 0.7940
|
| 13 |
+
ROC-AUC: 0.8824
|
| 14 |
+
PR-AUC: 0.9275
|
| 15 |
+
============================================================
|
| 16 |
+
|
| 17 |
+
Bootstrap 95% CI (n=1000):
|
| 18 |
+
Accuracy: [0.7862, 0.8223]
|
| 19 |
+
F1: [0.8245, 0.8571]
|
| 20 |
+
F1-macro: [0.7739, 0.8119]
|
| 21 |
+
ROC-AUC: [0.8667, 0.8969]
|
| 22 |
+
PR-AUC: [0.9141, 0.9391]
|
| 23 |
+
|
| 24 |
+
precision recall f1-score support
|
| 25 |
+
|
| 26 |
+
safe 0.71 0.78 0.75 719
|
| 27 |
+
unsafe 0.87 0.82 0.84 1245
|
| 28 |
+
|
| 29 |
+
accuracy 0.80 1964
|
| 30 |
+
macro avg 0.79 0.80 0.79 1964
|
| 31 |
+
weighted avg 0.81 0.80 0.81 1964
|