Upload nemotron-4B-think/mixed-ru/probes_quad/summary.txt with huggingface_hub
Browse files
nemotron-4B-think/mixed-ru/probes_quad/summary.txt
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
4-class probe: TP / TN / FP / FN
|
| 2 |
+
TP: 519
|
| 3 |
+
TN: 562
|
| 4 |
+
FP: 60
|
| 5 |
+
FN: 102
|
| 6 |
+
5-fold CV
|
| 7 |
+
|
| 8 |
+
============================================================
|
| 9 |
+
Top-5 layers by macro-F1
|
| 10 |
+
============================================================
|
| 11 |
+
Rank Layer F1-macro F1-weighted Acc
|
| 12 |
+
-------------------------------------------------------
|
| 13 |
+
1 18 0.8578+/-0.023 0.9352+/-0.012 0.9346+/-0.011
|
| 14 |
+
2 19 0.8552+/-0.035 0.9352+/-0.011 0.9356+/-0.009
|
| 15 |
+
3 20 0.8514+/-0.038 0.9388+/-0.012 0.9396+/-0.009
|
| 16 |
+
4 17 0.8457+/-0.045 0.9323+/-0.016 0.9326+/-0.015
|
| 17 |
+
5 16 0.8368+/-0.033 0.9210+/-0.017 0.9205+/-0.017
|
| 18 |
+
|
| 19 |
+
============================================================
|
| 20 |
+
Classification report (best layer 18, retrained on train data)
|
| 21 |
+
============================================================
|
| 22 |
+
precision recall f1-score support
|
| 23 |
+
|
| 24 |
+
FN 1.00 1.00 1.00 82
|
| 25 |
+
FP 1.00 1.00 1.00 48
|
| 26 |
+
TN 1.00 1.00 1.00 449
|
| 27 |
+
TP 1.00 1.00 1.00 415
|
| 28 |
+
|
| 29 |
+
accuracy 1.00 994
|
| 30 |
+
macro avg 1.00 1.00 1.00 994
|
| 31 |
+
weighted avg 1.00 1.00 1.00 994
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
============================================================
|
| 35 |
+
Held-out test set (249 samples)
|
| 36 |
+
============================================================
|
| 37 |
+
accuracy: 0.9438
|
| 38 |
+
f1_macro: 0.8945
|
| 39 |
+
f1_weighted: 0.9470
|
| 40 |
+
|
| 41 |
+
precision recall f1-score support
|
| 42 |
+
|
| 43 |
+
FN 0.66 0.95 0.78 20
|
| 44 |
+
FP 0.91 0.83 0.87 12
|
| 45 |
+
TN 1.00 0.96 0.98 113
|
| 46 |
+
TP 0.97 0.93 0.95 104
|
| 47 |
+
|
| 48 |
+
accuracy 0.94 249
|
| 49 |
+
macro avg 0.88 0.92 0.89 249
|
| 50 |
+
weighted avg 0.96 0.94 0.95 249
|