mp-juuuns commited on
Commit
7b8cd30
·
verified ·
1 Parent(s): 49f5d40

Add Matplotlib benchmark visualizations and source data

Browse files
.gitattributes CHANGED
@@ -1,4 +1,3 @@
1
  *.safetensors filter=lfs diff=lfs merge=lfs -text
2
- models/semeval-propaganda/seeds/seed42/tokenizer.json filter=lfs diff=lfs merge=lfs -text
3
- models/semeval-propaganda/seeds/seed43/tokenizer.json filter=lfs diff=lfs merge=lfs -text
4
- models/semeval-propaganda/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 
1
  *.safetensors filter=lfs diff=lfs merge=lfs -text
2
+ benchmark/figures/02_per_label_transfer_metrics.png filter=lfs diff=lfs merge=lfs -text
3
+ benchmark/figures/04_compression_ladder.png filter=lfs diff=lfs merge=lfs -text
 
README.md CHANGED
@@ -101,6 +101,12 @@ parameter counts, BF16 file sizes, hashes, and stage values are in
101
  [`benchmark/compression_ladder.csv`](benchmark/compression_ladder.csv) and
102
  [`provenance/task_agnostic_stages/`](provenance/task_agnostic_stages/).
103
 
 
 
 
 
 
 
104
  ## Downstream benchmark contract
105
 
106
  Because the root is headless, it cannot be scored as a classifier without
@@ -158,6 +164,12 @@ The task-agnostic KD initialization improved same-seed mean Macro-F1 by
158
  `+0.06939` over the structural-copy control. This is an observed comparison
159
  under this protocol, not a significance or universal-superiority claim.
160
 
 
 
 
 
 
 
161
  ### Task-agnostic base: per-label test results
162
 
163
  Precision, recall, and F1 are three-seed means. Support is the fixed number of
@@ -184,6 +196,11 @@ Per-seed values and sample SDs for all labels and all arms are available in
184
  [`benchmark/per_label_metrics.csv`](benchmark/per_label_metrics.csv) and
185
  [`benchmark/per_label_summary.csv`](benchmark/per_label_summary.csv).
186
 
 
 
 
 
 
187
  ### Efficiency and resource measurements
188
 
189
  These numbers belong to the fresh-head transfer checkpoints, not a standalone
@@ -203,6 +220,12 @@ about CPU, mobile, UNO Q, or other deployment performance. Full seed-level
203
  resource fields are in
204
  [`benchmark/resource_metrics.csv`](benchmark/resource_metrics.csv).
205
 
 
 
 
 
 
 
206
  ### Machine-readable benchmark bundle
207
 
208
  - [`benchmark/BENCHMARK_CARD.md`](benchmark/BENCHMARK_CARD.md): generated
@@ -220,6 +243,8 @@ resource fields are in
220
  parameters, CUDA memory, wall time, and test timing
221
  - [`benchmark/reports/`](benchmark/reports/): normalized full frozen reports,
222
  including article-level probability vectors
 
 
223
 
224
  ## Multilingual scope
225
 
 
101
  [`benchmark/compression_ladder.csv`](benchmark/compression_ladder.csv) and
102
  [`provenance/task_agnostic_stages/`](provenance/task_agnostic_stages/).
103
 
104
+ ![Task-free 24L to 4L compression ladder](benchmark/figures/04_compression_ladder.png)
105
+
106
+ The chart reports the actual unsharded safetensors parameter counts and BF16
107
+ file sizes. It does not imply that downstream quality scales monotonically with
108
+ layer count.
109
+
110
  ## Downstream benchmark contract
111
 
112
  Because the root is headless, it cannot be scored as a classifier without
 
164
  `+0.06939` over the structural-copy control. This is an observed comparison
165
  under this protocol, not a significance or universal-superiority claim.
166
 
167
+ ![Three-seed Macro-F1 and Micro-F1 comparison](benchmark/figures/01_overall_transfer_quality.png)
168
+
169
+ Bars are three-seed means and whiskers are sample SD. The specialized 4L point
170
+ is a separate historical lineage rather than a fine-tuned child of the root
171
+ base.
172
+
173
  ### Task-agnostic base: per-label test results
174
 
175
  Precision, recall, and F1 are three-seed means. Support is the fixed number of
 
196
  [`benchmark/per_label_metrics.csv`](benchmark/per_label_metrics.csv) and
197
  [`benchmark/per_label_summary.csv`](benchmark/per_label_summary.csv).
198
 
199
+ ![Per-label precision recall and F1](benchmark/figures/02_per_label_transfer_metrics.png)
200
+
201
+ Labels are ordered by their three-seed mean F1. Support remains visible because
202
+ several classes have only 7–14 positive test articles.
203
+
204
  ### Efficiency and resource measurements
205
 
206
  These numbers belong to the fresh-head transfer checkpoints, not a standalone
 
220
  resource fields are in
221
  [`benchmark/resource_metrics.csv`](benchmark/resource_metrics.csv).
222
 
223
+ ![Quality efficiency comparison](benchmark/figures/03_quality_efficiency.png)
224
+
225
+ These plots compare the recorded fresh-head transfer checkpoints only.
226
+ Apparent differences must not be generalized to the headless root on untested
227
+ hardware.
228
+
229
  ### Machine-readable benchmark bundle
230
 
231
  - [`benchmark/BENCHMARK_CARD.md`](benchmark/BENCHMARK_CARD.md): generated
 
243
  parameters, CUDA memory, wall time, and test timing
244
  - [`benchmark/reports/`](benchmark/reports/): normalized full frozen reports,
245
  including article-level probability vectors
246
+ - [`benchmark/figures/`](benchmark/figures/): Matplotlib PNG/PDF figures,
247
+ per-figure source CSVs, and a hash manifest
248
 
249
  ## Multilingual scope
250
 
SHA256SUMS CHANGED
@@ -1,9 +1,24 @@
1
  ee05350c7fda0af4da116020dd7b976d731de3beed4f5e717d5bcc95ead24550 .gitattributes
2
  77fd4710def9ec3c0f6225800e0235f15a425abd4a8b03559127fcd782612049 LICENSE
3
- 5fcf007de6b6e828711c874310de1e042f1dada6989c70930ac0881285a9dc61 README.md
4
  bf4073de38c3905effd2dcce6f38894f325c8a0a79bedfb2779f0545c86b9c9f benchmark/BENCHMARK_CARD.md
5
  e01bd7ac59a3212e44b77e5fcfbc7c41ec91d74c998306d2438209d6ef1a3aa7 benchmark/arm_summary.csv
6
  2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b benchmark/compression_ladder.csv
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  ffc21be32b2106c7bf1df3854097a975ac370da01ac1b732eb5c2ba9075ddeda benchmark/per_label_metrics.csv
8
  c8e3876abf6cfe960c9332d3bc9d1fbde2f3213f3111351acf83b21bbb184049 benchmark/per_label_summary.csv
9
  c1bd379680abcf50b47b7d23e4aafbc8b5e7f40a0b0e9700bde21a8ce49af803 benchmark/reports/existing_specialized_separate_lineage-seed41.json
@@ -70,7 +85,7 @@ ec734855870758cbfef99031dac2ded85b54c496532e5c514a9a4f63287c3493 provenance/bas
70
  b9ce99b993c4f4563bc2521fc731bfd82f5e8149c141133599fbd8f982ba43d9 provenance/task_agnostic_stages/24to8.json
71
  ec734855870758cbfef99031dac2ded85b54c496532e5c514a9a4f63287c3493 provenance/task_agnostic_stages/6to4.json
72
  65e5383afbbcfe0b7dd78a5411c319ed6bf2586659fb1892aca9ecbd5dfd4993 provenance/task_agnostic_stages/8to6.json
73
- 207e1df5765c838b7d355a0c34cf5ae2c55f90b419cff0f4bbf80528a48fd939 release_manifest.json
74
  7b48e7811e56e552fc4aef44189e1d7c9b1352c85d2b3f10ff0bdd1ea5cdd824 requirements.txt
75
  06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523 tokenizer.json
76
  5ab9bed0a4d27949672f65ba1141d6dd5b0514fb9091d3426506ecebb5d5e294 tokenizer_config.json
 
1
  ee05350c7fda0af4da116020dd7b976d731de3beed4f5e717d5bcc95ead24550 .gitattributes
2
  77fd4710def9ec3c0f6225800e0235f15a425abd4a8b03559127fcd782612049 LICENSE
3
+ 49cb7082e65f26835c90131ce7ff05f47ebd77ec9352dad491a0083ede70bd50 README.md
4
  bf4073de38c3905effd2dcce6f38894f325c8a0a79bedfb2779f0545c86b9c9f benchmark/BENCHMARK_CARD.md
5
  e01bd7ac59a3212e44b77e5fcfbc7c41ec91d74c998306d2438209d6ef1a3aa7 benchmark/arm_summary.csv
6
  2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b benchmark/compression_ladder.csv
7
+ 769a516e53417e16b61cf30d95a02f6c812c0b1536242cdbf836da779753efbd benchmark/figures/01_overall_transfer_quality.csv
8
+ 7221e0d2feb356e4d7f44aa0381e576254f734837e3f54fb0dfb98795ec19ded benchmark/figures/01_overall_transfer_quality.pdf
9
+ 6d35b5964791cfff2701e43831996a9aad7d607305ab3031b0782bed2ffffa66 benchmark/figures/01_overall_transfer_quality.png
10
+ 50a3ceb0e3146ba8cade9c86c4b59c7db6cbedfc41fac39a811d7f39f4bdeb0d benchmark/figures/02_per_label_transfer_metrics.csv
11
+ 10756816e8ad800f5d872907986680abf637e76fe47f9790f27645f8f231ae4a benchmark/figures/02_per_label_transfer_metrics.pdf
12
+ 5b558738b405a7b04bb30f38b6e53af21b2392ea12cad0d777e9d6d6985d3d04 benchmark/figures/02_per_label_transfer_metrics.png
13
+ f2a34cbcd700b2663cf761a5fb1dbd3aa99f310a96d4b4477a3a4529bee1a1b3 benchmark/figures/03_quality_efficiency.csv
14
+ 18c81ce50fc7871b000c9e387f0f7d7780002cb2bd249153aadfa55a186342c0 benchmark/figures/03_quality_efficiency.pdf
15
+ 08ed7ca8193d5f9065645fb22491ab88ee260d43566e9013aa0f98d7ab4dde87 benchmark/figures/03_quality_efficiency.png
16
+ 2dfac2b8330724ce46a22d8f3ddfeffc767f52b93a93cb1a833c89c5820ffccc benchmark/figures/04_compression_ladder.csv
17
+ 953d6e356498127b2a607530fac2e41b07b4bf38ed0af945f778b784dc6f7b23 benchmark/figures/04_compression_ladder.pdf
18
+ 9f6c1d6fdefd45711fcfa57a7d0d91f6baaa3a5ee0b44a36554a9c74febe7090 benchmark/figures/04_compression_ladder.png
19
+ 7e00107e6576da117689dca0c164c497ce32e9d93c034331e2e161625afc2092 benchmark/figures/README.md
20
+ b3e3da2fcf1ae8ea9c88019c289e3d45e7945df655218d5c0d3b905769811f50 benchmark/figures/manifest.json
21
+ 3d2c9fcf9c6cc7da1a885968b70a42eaa38659d317c9d3290a2b3066f3f9ee6a benchmark/figures/source_summary.json
22
  ffc21be32b2106c7bf1df3854097a975ac370da01ac1b732eb5c2ba9075ddeda benchmark/per_label_metrics.csv
23
  c8e3876abf6cfe960c9332d3bc9d1fbde2f3213f3111351acf83b21bbb184049 benchmark/per_label_summary.csv
24
  c1bd379680abcf50b47b7d23e4aafbc8b5e7f40a0b0e9700bde21a8ce49af803 benchmark/reports/existing_specialized_separate_lineage-seed41.json
 
85
  b9ce99b993c4f4563bc2521fc731bfd82f5e8149c141133599fbd8f982ba43d9 provenance/task_agnostic_stages/24to8.json
86
  ec734855870758cbfef99031dac2ded85b54c496532e5c514a9a4f63287c3493 provenance/task_agnostic_stages/6to4.json
87
  65e5383afbbcfe0b7dd78a5411c319ed6bf2586659fb1892aca9ecbd5dfd4993 provenance/task_agnostic_stages/8to6.json
88
+ a37047ec47182dba0a5d9c455f8d3f1d4ec852b67e3229301802058f973e2ebf release_manifest.json
89
  7b48e7811e56e552fc4aef44189e1d7c9b1352c85d2b3f10ff0bdd1ea5cdd824 requirements.txt
90
  06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523 tokenizer.json
91
  5ab9bed0a4d27949672f65ba1141d6dd5b0514fb9091d3426506ecebb5d5e294 tokenizer_config.json
benchmark/figures/01_overall_transfer_quality.csv ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ arm,metric,mean,sample_sd,n_seeds
2
+ task_agnostic_base,macro_f1,0.5874762842563245,0.01579462678670608,3
3
+ task_agnostic_base,micro_f1,0.6529504772586388,0.010884055708931178,3
4
+ structural_copy_control,macro_f1,0.5180869574433028,0.016311512588628545,3
5
+ structural_copy_control,micro_f1,0.5904584952122559,0.01602830914762526,3
6
+ existing_specialized_separate_lineage,macro_f1,0.5875679559555907,0.007165753447433348,3
7
+ existing_specialized_separate_lineage,micro_f1,0.647978512363484,0.0018739735696546695,3
benchmark/figures/01_overall_transfer_quality.pdf ADDED
Binary file (18.1 kB). View file
 
benchmark/figures/01_overall_transfer_quality.png ADDED
benchmark/figures/02_per_label_transfer_metrics.csv ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ label,support,precision_mean,recall_mean,f1_mean
2
+ Thought-terminating_Cliches,7,0.3492063492063492,0.3333333333333333,0.319014319014319
3
+ "Whataboutism,Straw_Men,Red_Herring",10,0.2761904761904762,0.5,0.330448343079922
4
+ Black-and-White_Fallacy,12,0.3667929292929293,0.47222222222222227,0.4053892966936445
5
+ Appeal_to_Authority,11,0.3958614484930274,0.5757575757575757,0.4657088122605364
6
+ "Exaggeration,Minimisation",22,0.4620811287477954,0.7727272727272727,0.5727040816326531
7
+ Slogans,14,0.5161361626878869,0.7380952380952381,0.5834824090638044
8
+ "Bandwagon,Reductio_ad_hitlerum",7,0.4851851851851852,0.8095238095238095,0.6006778309409889
9
+ Repetition,23,0.5196620583717357,0.7536231884057971,0.6147520781519571
10
+ Appeal_to_fear-prejudice,24,0.5540244550770866,0.7361111111111112,0.6277521761392729
11
+ Causal_Oversimplification,18,0.5428824049513704,0.7962962962962963,0.6446992723588468
12
+ Flag-Waving,17,0.5642642642642642,0.8235294117647058,0.6464506172839506
13
+ Doubt,22,0.5223832528180354,0.9848484848484849,0.6802057822343391
14
+ "Name_Calling,Labeling",33,0.7380952380952381,0.9090909090909091,0.8125087983911513
15
+ Loaded_Language,45,0.8649032907793016,0.9851851851851852,0.9208741623431562
benchmark/figures/02_per_label_transfer_metrics.pdf ADDED
Binary file (21.4 kB). View file
 
benchmark/figures/02_per_label_transfer_metrics.png ADDED

Git LFS Details

  • SHA256: 5b558738b405a7b04bb30f38b6e53af21b2392ea12cad0d777e9d6d6985d3d04
  • Pointer size: 131 Bytes
  • Size of remote file: 138 kB
benchmark/figures/03_quality_efficiency.csv ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ arm,macro_f1_mean,macro_f1_sample_sd,test_ms_per_article_mean,peak_cuda_allocated_gib_mean,parameters_mean
2
+ task_agnostic_base,0.5874762842563245,0.01579462678670608,81.30633847886752,3.1487789154052734,334111040
3
+ structural_copy_control,0.5180869574433028,0.016311512588628545,81.16211107893226,3.1487789154052734,334111040
4
+ existing_specialized_separate_lineage,0.5875679559555907,0.007165753447433348,85.43555729062946,1.9964351654052734,210903360
benchmark/figures/03_quality_efficiency.pdf ADDED
Binary file (21.9 kB). View file
 
benchmark/figures/03_quality_efficiency.png ADDED
benchmark/figures/04_compression_ladder.csv ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ name,layers,parameters_millions,weight_mib,kd_stage,mean_total_loss
2
+ 24L upstream text teacher,24,752.393024,1435.1135559082031,,
3
+ 8L task-free KD,8,420.318144,801.7042922973633,24to8,0.17622653172702485
4
+ 6L task-free KD,6,377.207424,719.4742431640625,8to6,0.03382104352249371
5
+ 4L task-free KD root,4,334.096704,637.2442016601562,6to4,0.047020394468859195
benchmark/figures/04_compression_ladder.pdf ADDED
Binary file (19.5 kB). View file
 
benchmark/figures/04_compression_ladder.png ADDED

Git LFS Details

  • SHA256: 9f6c1d6fdefd45711fcfa57a7d0d91f6baaa3a5ee0b44a36554a9c74febe7090
  • Pointer size: 131 Bytes
  • Size of remote file: 121 kB
benchmark/figures/README.md ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Matplotlib benchmark figures
2
+
3
+ All figures are deterministically rendered from the frozen v2 benchmark bundle.
4
+ PNG files are intended for the Hugging Face model card; PDF files are vector
5
+ exports for reports. Each plot has a compact source CSV beside it.
6
+
7
+ 1. Overall Macro/Micro-F1: three-seed means with sample-SD whiskers.
8
+ 2. Per-label precision/recall/F1 for the task-agnostic base.
9
+ 3. Macro-F1 against article inference time and peak CUDA allocation.
10
+ 4. Parameters and BF16 size across the 24L→8L→6L→4L task-free ladder.
11
+
12
+ The 55-article test split was previously opened. Error bars are not confidence
13
+ intervals. The specialized checkpoint is a separate reference lineage.
benchmark/figures/manifest.json ADDED
@@ -0,0 +1,84 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "schema": "standalone4l-matplotlib-figures-v1",
3
+ "status": "complete",
4
+ "source_benchmark_schema": "standalone4l-complete-transfer-benchmark-v2",
5
+ "source_summary_sha256": "3d2c9fcf9c6cc7da1a885968b70a42eaa38659d317c9d3290a2b3066f3f9ee6a",
6
+ "figures": [
7
+ "01_overall_transfer_quality",
8
+ "02_per_label_transfer_metrics",
9
+ "03_quality_efficiency",
10
+ "04_compression_ladder"
11
+ ],
12
+ "files_before_manifest": [
13
+ {
14
+ "path": "01_overall_transfer_quality.csv",
15
+ "bytes": 500,
16
+ "sha256": "769a516e53417e16b61cf30d95a02f6c812c0b1536242cdbf836da779753efbd"
17
+ },
18
+ {
19
+ "path": "01_overall_transfer_quality.pdf",
20
+ "bytes": 18071,
21
+ "sha256": "7221e0d2feb356e4d7f44aa0381e576254f734837e3f54fb0dfb98795ec19ded"
22
+ },
23
+ {
24
+ "path": "01_overall_transfer_quality.png",
25
+ "bytes": 82127,
26
+ "sha256": "6d35b5964791cfff2701e43831996a9aad7d607305ab3031b0782bed2ffffa66"
27
+ },
28
+ {
29
+ "path": "02_per_label_transfer_metrics.csv",
30
+ "bytes": 1169,
31
+ "sha256": "50a3ceb0e3146ba8cade9c86c4b59c7db6cbedfc41fac39a811d7f39f4bdeb0d"
32
+ },
33
+ {
34
+ "path": "02_per_label_transfer_metrics.pdf",
35
+ "bytes": 21379,
36
+ "sha256": "10756816e8ad800f5d872907986680abf637e76fe47f9790f27645f8f231ae4a"
37
+ },
38
+ {
39
+ "path": "02_per_label_transfer_metrics.png",
40
+ "bytes": 138298,
41
+ "sha256": "5b558738b405a7b04bb30f38b6e53af21b2392ea12cad0d777e9d6d6985d3d04"
42
+ },
43
+ {
44
+ "path": "03_quality_efficiency.csv",
45
+ "bytes": 448,
46
+ "sha256": "f2a34cbcd700b2663cf761a5fb1dbd3aa99f310a96d4b4477a3a4529bee1a1b3"
47
+ },
48
+ {
49
+ "path": "03_quality_efficiency.pdf",
50
+ "bytes": 21933,
51
+ "sha256": "18c81ce50fc7871b000c9e387f0f7d7780002cb2bd249153aadfa55a186342c0"
52
+ },
53
+ {
54
+ "path": "03_quality_efficiency.png",
55
+ "bytes": 99374,
56
+ "sha256": "08ed7ca8193d5f9065645fb22491ab88ee260d43566e9013aa0f98d7ab4dde87"
57
+ },
58
+ {
59
+ "path": "04_compression_ladder.csv",
60
+ "bytes": 352,
61
+ "sha256": "2dfac2b8330724ce46a22d8f3ddfeffc767f52b93a93cb1a833c89c5820ffccc"
62
+ },
63
+ {
64
+ "path": "04_compression_ladder.pdf",
65
+ "bytes": 19490,
66
+ "sha256": "953d6e356498127b2a607530fac2e41b07b4bf38ed0af945f778b784dc6f7b23"
67
+ },
68
+ {
69
+ "path": "04_compression_ladder.png",
70
+ "bytes": 120809,
71
+ "sha256": "9f6c1d6fdefd45711fcfa57a7d0d91f6baaa3a5ee0b44a36554a9c74febe7090"
72
+ },
73
+ {
74
+ "path": "README.md",
75
+ "bytes": 683,
76
+ "sha256": "7e00107e6576da117689dca0c164c497ce32e9d93c034331e2e161625afc2092"
77
+ },
78
+ {
79
+ "path": "source_summary.json",
80
+ "bytes": 26913,
81
+ "sha256": "3d2c9fcf9c6cc7da1a885968b70a42eaa38659d317c9d3290a2b3066f3f9ee6a"
82
+ }
83
+ ]
84
+ }
benchmark/figures/source_summary.json ADDED
@@ -0,0 +1,891 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "schema": "standalone4l-complete-transfer-benchmark-v2",
3
+ "status": "complete_from_frozen_reports",
4
+ "claim_scope": "Exploratory fresh-head transfer evidence on one English task and a previously opened 55-article SemEval-derived test split.",
5
+ "headless_root_requires_adaptation": true,
6
+ "test_previously_opened": true,
7
+ "test_articles": 55,
8
+ "test_windows": 434,
9
+ "seeds": [
10
+ 41,
11
+ 42,
12
+ 43
13
+ ],
14
+ "three_seed_sd_is_not_confidence_interval": true,
15
+ "multilingual_downstream_evaluation_performed": false,
16
+ "upstream_multilingual_metadata": {
17
+ "source": "Qwen/Qwen3.5-0.8B official model card",
18
+ "languages_and_dialects_claimed_upstream": 201,
19
+ "scope": "inherited tokenizer and architecture coverage only"
20
+ },
21
+ "protocol": {
22
+ "epochs": 5,
23
+ "primary_epoch": 5,
24
+ "early_stopping": false,
25
+ "max_length": 256,
26
+ "stride": 128,
27
+ "window_target": "span-aligned technique labels",
28
+ "article_aggregation": "labelwise maximum probability",
29
+ "micro_batch_size": 1,
30
+ "gradient_accumulation": 32,
31
+ "effective_batch_size": 32,
32
+ "learning_rate": 2e-05,
33
+ "weight_decay": 0.01,
34
+ "gradient_clip": 1.0,
35
+ "optimizer": "AdamW",
36
+ "dtype": "bfloat16",
37
+ "thresholds": [
38
+ 0.05,
39
+ 0.1,
40
+ 0.15,
41
+ 0.2,
42
+ 0.25,
43
+ 0.3,
44
+ 0.35,
45
+ 0.4,
46
+ 0.45,
47
+ 0.5,
48
+ 0.55,
49
+ 0.6,
50
+ 0.65,
51
+ 0.7,
52
+ 0.75,
53
+ 0.8,
54
+ 0.85,
55
+ 0.9
56
+ ],
57
+ "threshold_selection": "epoch-5 calibration macro-F1, then micro-F1"
58
+ },
59
+ "arms": {
60
+ "task_agnostic_base": {
61
+ "display": "Task-agnostic KD base",
62
+ "lineage": "root base plus a fresh 14-label transfer head",
63
+ "metrics": {
64
+ "macro_f1": {
65
+ "values": [
66
+ 0.5883093293517536,
67
+ 0.6028379036218096,
68
+ 0.5712816197954101
69
+ ],
70
+ "mean": 0.5874762842563245,
71
+ "sample_sd": 0.01579462678670608,
72
+ "n": 3
73
+ },
74
+ "micro_f1": {
75
+ "values": [
76
+ 0.6468085106382978,
77
+ 0.6655172413793103,
78
+ 0.6465256797583082
79
+ ],
80
+ "mean": 0.6529504772586388,
81
+ "sample_sd": 0.010884055708931178,
82
+ "n": 3
83
+ },
84
+ "exact_match": {
85
+ "values": [
86
+ 0.05454545454545454,
87
+ 0.03636363636363636,
88
+ 0.0
89
+ ],
90
+ "mean": 0.0303030303030303,
91
+ "sample_sd": 0.027773186030035393,
92
+ "n": 3
93
+ },
94
+ "predicted_positive_rate": {
95
+ "values": [
96
+ 0.5714285714285714,
97
+ 0.4090909090909091,
98
+ 0.5155844155844156
99
+ ],
100
+ "mean": 0.4987012987012987,
101
+ "sample_sd": 0.08247520160679181,
102
+ "n": 3
103
+ }
104
+ },
105
+ "resources": {
106
+ "parameters": {
107
+ "values": [
108
+ 334111040.0,
109
+ 334111040.0,
110
+ 334111040.0
111
+ ],
112
+ "mean": 334111040.0,
113
+ "sample_sd": 0.0,
114
+ "n": 3
115
+ },
116
+ "peak_cuda_allocated_bytes": {
117
+ "values": [
118
+ 3380975616.0,
119
+ 3380975616.0,
120
+ 3380975616.0
121
+ ],
122
+ "mean": 3380975616.0,
123
+ "sample_sd": 0.0,
124
+ "n": 3
125
+ },
126
+ "peak_cuda_reserved_bytes": {
127
+ "values": [
128
+ 3519021056.0,
129
+ 3519021056.0,
130
+ 3510632448.0
131
+ ],
132
+ "mean": 3516224853.3333335,
133
+ "sample_sd": 4843165.086926248,
134
+ "n": 3
135
+ },
136
+ "total_wall_seconds": {
137
+ "values": [
138
+ 1583.0297977919981,
139
+ 1588.617994428001,
140
+ 1603.4310500120046
141
+ ],
142
+ "mean": 1591.692947410668,
143
+ "sample_sd": 10.54249851412311,
144
+ "n": 3
145
+ },
146
+ "test_seconds_per_article": {
147
+ "values": [
148
+ 0.08164824154545468,
149
+ 0.07871871667289683,
150
+ 0.08355205721825107
151
+ ],
152
+ "mean": 0.08130633847886752,
153
+ "sample_sd": 0.0024347419754560395,
154
+ "n": 3
155
+ }
156
+ },
157
+ "thresholds": [
158
+ 0.15,
159
+ 0.3,
160
+ 0.1
161
+ ]
162
+ },
163
+ "structural_copy_control": {
164
+ "display": "Structural copy, no task-free KD",
165
+ "lineage": "same four-layer structure without task-free KD",
166
+ "metrics": {
167
+ "macro_f1": {
168
+ "values": [
169
+ 0.5169424579262757,
170
+ 0.5349405779540175,
171
+ 0.5023778364496152
172
+ ],
173
+ "mean": 0.5180869574433028,
174
+ "sample_sd": 0.016311512588628545,
175
+ "n": 3
176
+ },
177
+ "micro_f1": {
178
+ "values": [
179
+ 0.5773955773955775,
180
+ 0.6083445491251682,
181
+ 0.585635359116022
182
+ ],
183
+ "mean": 0.5904584952122559,
184
+ "sample_sd": 0.01602830914762526,
185
+ "n": 3
186
+ },
187
+ "exact_match": {
188
+ "values": [
189
+ 0.0,
190
+ 0.0,
191
+ 0.0
192
+ ],
193
+ "mean": 0.0,
194
+ "sample_sd": 0.0,
195
+ "n": 3
196
+ },
197
+ "predicted_positive_rate": {
198
+ "values": [
199
+ 0.712987012987013,
200
+ 0.6207792207792208,
201
+ 0.5961038961038961
202
+ ],
203
+ "mean": 0.6432900432900432,
204
+ "sample_sd": 0.06160737456395461,
205
+ "n": 3
206
+ }
207
+ },
208
+ "resources": {
209
+ "parameters": {
210
+ "values": [
211
+ 334111040.0,
212
+ 334111040.0,
213
+ 334111040.0
214
+ ],
215
+ "mean": 334111040.0,
216
+ "sample_sd": 0.0,
217
+ "n": 3
218
+ },
219
+ "peak_cuda_allocated_bytes": {
220
+ "values": [
221
+ 3380975616.0,
222
+ 3380975616.0,
223
+ 3380975616.0
224
+ ],
225
+ "mean": 3380975616.0,
226
+ "sample_sd": 0.0,
227
+ "n": 3
228
+ },
229
+ "peak_cuda_reserved_bytes": {
230
+ "values": [
231
+ 3519021056.0,
232
+ 3519021056.0,
233
+ 3510632448.0
234
+ ],
235
+ "mean": 3516224853.3333335,
236
+ "sample_sd": 4843165.086926248,
237
+ "n": 3
238
+ },
239
+ "total_wall_seconds": {
240
+ "values": [
241
+ 1575.5905925639963,
242
+ 1623.5639530190092,
243
+ 1629.195114912989
244
+ ],
245
+ "mean": 1609.4498868319981,
246
+ "sample_sd": 29.457874198835384,
247
+ "n": 3
248
+ },
249
+ "test_seconds_per_article": {
250
+ "values": [
251
+ 0.07877608058201573,
252
+ 0.08307423260041767,
253
+ 0.08163602005436339
254
+ ],
255
+ "mean": 0.08116211107893226,
256
+ "sample_sd": 0.002187914527825257,
257
+ "n": 3
258
+ }
259
+ },
260
+ "thresholds": [
261
+ 0.05,
262
+ 0.2,
263
+ 0.2
264
+ ]
265
+ },
266
+ "existing_specialized_separate_lineage": {
267
+ "display": "Existing specialized 4L, separate lineage",
268
+ "lineage": "historical task-specific shrink/distillation lineage",
269
+ "metrics": {
270
+ "macro_f1": {
271
+ "values": [
272
+ 0.5832848452288824,
273
+ 0.5958405181160762,
274
+ 0.5835785045218137
275
+ ],
276
+ "mean": 0.5875679559555907,
277
+ "sample_sd": 0.007165753447433348,
278
+ "n": 3
279
+ },
280
+ "micro_f1": {
281
+ "values": [
282
+ 0.6494252873563218,
283
+ 0.6458616010854816,
284
+ 0.6486486486486486
285
+ ],
286
+ "mean": 0.647978512363484,
287
+ "sample_sd": 0.0018739735696546695,
288
+ "n": 3
289
+ },
290
+ "exact_match": {
291
+ "values": [
292
+ 0.0,
293
+ 0.0,
294
+ 0.01818181818181818
295
+ ],
296
+ "mean": 0.006060606060606061,
297
+ "sample_sd": 0.01049727762162956,
298
+ "n": 3
299
+ },
300
+ "predicted_positive_rate": {
301
+ "values": [
302
+ 0.5597402597402598,
303
+ 0.612987012987013,
304
+ 0.5688311688311688
305
+ ],
306
+ "mean": 0.5805194805194805,
307
+ "sample_sd": 0.02848274311618351,
308
+ "n": 3
309
+ }
310
+ },
311
+ "resources": {
312
+ "parameters": {
313
+ "values": [
314
+ 210903360.0,
315
+ 210903360.0,
316
+ 210903360.0
317
+ ],
318
+ "mean": 210903360.0,
319
+ "sample_sd": 0.0,
320
+ "n": 3
321
+ },
322
+ "peak_cuda_allocated_bytes": {
323
+ "values": [
324
+ 2143655936.0,
325
+ 2143655936.0,
326
+ 2143655936.0
327
+ ],
328
+ "mean": 2143655936.0,
329
+ "sample_sd": 0.0,
330
+ "n": 3
331
+ },
332
+ "peak_cuda_reserved_bytes": {
333
+ "values": [
334
+ 2281701376.0,
335
+ 2281701376.0,
336
+ 2273312768.0
337
+ ],
338
+ "mean": 2278905173.3333335,
339
+ "sample_sd": 4843165.086926248,
340
+ "n": 3
341
+ },
342
+ "total_wall_seconds": {
343
+ "values": [
344
+ 1634.488557530014,
345
+ 1652.9397718419787,
346
+ 1643.488202479988
347
+ ],
348
+ "mean": 1643.6388439506602,
349
+ "sample_sd": 9.226529522850198,
350
+ "n": 3
351
+ },
352
+ "test_seconds_per_article": {
353
+ "values": [
354
+ 0.0847959352360869,
355
+ 0.08598086021798239,
356
+ 0.08552987641781908
357
+ ],
358
+ "mean": 0.08543555729062946,
359
+ "sample_sd": 0.0005980667826383295,
360
+ "n": 3
361
+ }
362
+ },
363
+ "thresholds": [
364
+ 0.1,
365
+ 0.05,
366
+ 0.05
367
+ ]
368
+ }
369
+ },
370
+ "same_seed_macro_f1_deltas_base_minus_control": [
371
+ 0.07136687142547793,
372
+ 0.06789732566779205,
373
+ 0.06890378334579494
374
+ ],
375
+ "mean_macro_f1_delta_base_minus_control": 0.06938932681302164,
376
+ "publication_gate_passed": true,
377
+ "per_label_summary": [
378
+ {
379
+ "arm": "task_agnostic_base",
380
+ "label": "Appeal_to_Authority",
381
+ "support": 11,
382
+ "precision_mean": 0.3958614484930274,
383
+ "precision_sample_sd": 0.1238064457010119,
384
+ "recall_mean": 0.5757575757575757,
385
+ "recall_sample_sd": 0.05248638810814781,
386
+ "f1_mean": 0.4657088122605364,
387
+ "f1_sample_sd": 0.1020990126979069
388
+ },
389
+ {
390
+ "arm": "task_agnostic_base",
391
+ "label": "Appeal_to_fear-prejudice",
392
+ "support": 24,
393
+ "precision_mean": 0.5540244550770866,
394
+ "precision_sample_sd": 0.02525673990402998,
395
+ "recall_mean": 0.7361111111111112,
396
+ "recall_sample_sd": 0.13393959390267993,
397
+ "f1_mean": 0.6277521761392729,
398
+ "f1_sample_sd": 0.04994879016183189
399
+ },
400
+ {
401
+ "arm": "task_agnostic_base",
402
+ "label": "Bandwagon,Reductio_ad_hitlerum",
403
+ "support": 7,
404
+ "precision_mean": 0.4851851851851852,
405
+ "precision_sample_sd": 0.07882887649552875,
406
+ "recall_mean": 0.8095238095238095,
407
+ "recall_sample_sd": 0.08247860988423221,
408
+ "f1_mean": 0.6006778309409889,
409
+ "f1_sample_sd": 0.04793776258918466
410
+ },
411
+ {
412
+ "arm": "task_agnostic_base",
413
+ "label": "Black-and-White_Fallacy",
414
+ "support": 12,
415
+ "precision_mean": 0.3667929292929293,
416
+ "precision_sample_sd": 0.07670649392711647,
417
+ "recall_mean": 0.47222222222222227,
418
+ "recall_sample_sd": 0.09622504486493764,
419
+ "f1_mean": 0.4053892966936445,
420
+ "f1_sample_sd": 0.04211369218048746
421
+ },
422
+ {
423
+ "arm": "task_agnostic_base",
424
+ "label": "Causal_Oversimplification",
425
+ "support": 18,
426
+ "precision_mean": 0.5428824049513704,
427
+ "precision_sample_sd": 0.039216350192122156,
428
+ "recall_mean": 0.7962962962962963,
429
+ "recall_sample_sd": 0.11564811108145183,
430
+ "f1_mean": 0.6446992723588468,
431
+ "f1_sample_sd": 0.0634561308339175
432
+ },
433
+ {
434
+ "arm": "task_agnostic_base",
435
+ "label": "Doubt",
436
+ "support": 22,
437
+ "precision_mean": 0.5223832528180354,
438
+ "precision_sample_sd": 0.06742780050556177,
439
+ "recall_mean": 0.9848484848484849,
440
+ "recall_sample_sd": 0.026243194054073875,
441
+ "f1_mean": 0.6802057822343391,
442
+ "f1_sample_sd": 0.04928561754686331
443
+ },
444
+ {
445
+ "arm": "task_agnostic_base",
446
+ "label": "Exaggeration,Minimisation",
447
+ "support": 22,
448
+ "precision_mean": 0.4620811287477954,
449
+ "precision_sample_sd": 0.016801198309751,
450
+ "recall_mean": 0.7727272727272727,
451
+ "recall_sample_sd": 0.15745916432444337,
452
+ "f1_mean": 0.5727040816326531,
453
+ "f1_sample_sd": 0.03645259990419198
454
+ },
455
+ {
456
+ "arm": "task_agnostic_base",
457
+ "label": "Flag-Waving",
458
+ "support": 17,
459
+ "precision_mean": 0.5642642642642642,
460
+ "precision_sample_sd": 0.1478145677656334,
461
+ "recall_mean": 0.8235294117647058,
462
+ "recall_sample_sd": 0.1764705882352941,
463
+ "f1_mean": 0.6464506172839506,
464
+ "f1_sample_sd": 0.035742219952758225
465
+ },
466
+ {
467
+ "arm": "task_agnostic_base",
468
+ "label": "Loaded_Language",
469
+ "support": 45,
470
+ "precision_mean": 0.8649032907793016,
471
+ "precision_sample_sd": 0.046984312428427724,
472
+ "recall_mean": 0.9851851851851852,
473
+ "recall_sample_sd": 0.012830005981991702,
474
+ "f1_mean": 0.9208741623431562,
475
+ "f1_sample_sd": 0.03200926878737141
476
+ },
477
+ {
478
+ "arm": "task_agnostic_base",
479
+ "label": "Name_Calling,Labeling",
480
+ "support": 33,
481
+ "precision_mean": 0.7380952380952381,
482
+ "precision_sample_sd": 0.0047619047619048005,
483
+ "recall_mean": 0.9090909090909091,
484
+ "recall_sample_sd": 0.10925912955951485,
485
+ "f1_mean": 0.8125087983911513,
486
+ "f1_sample_sd": 0.04252971464057227
487
+ },
488
+ {
489
+ "arm": "task_agnostic_base",
490
+ "label": "Repetition",
491
+ "support": 23,
492
+ "precision_mean": 0.5196620583717357,
493
+ "precision_sample_sd": 0.021645909050194662,
494
+ "recall_mean": 0.7536231884057971,
495
+ "recall_sample_sd": 0.06641414050660639,
496
+ "f1_mean": 0.6147520781519571,
497
+ "f1_sample_sd": 0.03505927109416221
498
+ },
499
+ {
500
+ "arm": "task_agnostic_base",
501
+ "label": "Slogans",
502
+ "support": 14,
503
+ "precision_mean": 0.5161361626878869,
504
+ "precision_sample_sd": 0.20305677930281646,
505
+ "recall_mean": 0.7380952380952381,
506
+ "recall_sample_sd": 0.10910894511799614,
507
+ "f1_mean": 0.5834824090638044,
508
+ "f1_sample_sd": 0.09862686289220693
509
+ },
510
+ {
511
+ "arm": "task_agnostic_base",
512
+ "label": "Thought-terminating_Cliches",
513
+ "support": 7,
514
+ "precision_mean": 0.3492063492063492,
515
+ "precision_sample_sd": 0.143516994603374,
516
+ "recall_mean": 0.3333333333333333,
517
+ "recall_sample_sd": 0.08247860988423225,
518
+ "f1_mean": 0.319014319014319,
519
+ "f1_sample_sd": 0.04017590711117064
520
+ },
521
+ {
522
+ "arm": "task_agnostic_base",
523
+ "label": "Whataboutism,Straw_Men,Red_Herring",
524
+ "support": 10,
525
+ "precision_mean": 0.2761904761904762,
526
+ "precision_sample_sd": 0.05302632726504782,
527
+ "recall_mean": 0.5,
528
+ "recall_sample_sd": 0.2645751311064591,
529
+ "f1_mean": 0.330448343079922,
530
+ "f1_sample_sd": 0.021845164720973014
531
+ },
532
+ {
533
+ "arm": "structural_copy_control",
534
+ "label": "Appeal_to_Authority",
535
+ "support": 11,
536
+ "precision_mean": 0.26003086419753085,
537
+ "precision_sample_sd": 0.06281357095350094,
538
+ "recall_mean": 0.5757575757575758,
539
+ "recall_sample_sd": 0.262431940540739,
540
+ "f1_mean": 0.35477582846003897,
541
+ "f1_sample_sd": 0.11479479036518875
542
+ },
543
+ {
544
+ "arm": "structural_copy_control",
545
+ "label": "Appeal_to_fear-prejudice",
546
+ "support": 24,
547
+ "precision_mean": 0.5104813315339631,
548
+ "precision_sample_sd": 0.038427508996734555,
549
+ "recall_mean": 0.8611111111111112,
550
+ "recall_sample_sd": 0.024056261216234387,
551
+ "f1_mean": 0.6401745363877202,
552
+ "f1_sample_sd": 0.029305467347150463
553
+ },
554
+ {
555
+ "arm": "structural_copy_control",
556
+ "label": "Bandwagon,Reductio_ad_hitlerum",
557
+ "support": 7,
558
+ "precision_mean": 0.2876623376623376,
559
+ "precision_sample_sd": 0.061386823077618426,
560
+ "recall_mean": 0.8571428571428571,
561
+ "recall_sample_sd": 0.14285714285714285,
562
+ "f1_mean": 0.4306391777656145,
563
+ "f1_sample_sd": 0.08686392624022733
564
+ },
565
+ {
566
+ "arm": "structural_copy_control",
567
+ "label": "Black-and-White_Fallacy",
568
+ "support": 12,
569
+ "precision_mean": 0.31339031339031337,
570
+ "precision_sample_sd": 0.08301311842925053,
571
+ "recall_mean": 0.5277777777777778,
572
+ "recall_sample_sd": 0.2678791878053599,
573
+ "f1_mean": 0.38778427550357375,
574
+ "f1_sample_sd": 0.13069755090515112
575
+ },
576
+ {
577
+ "arm": "structural_copy_control",
578
+ "label": "Causal_Oversimplification",
579
+ "support": 18,
580
+ "precision_mean": 0.4464201043148412,
581
+ "precision_sample_sd": 0.013538446216471536,
582
+ "recall_mean": 0.9259259259259259,
583
+ "recall_sample_sd": 0.03207501495497923,
584
+ "f1_mean": 0.6023809523809524,
585
+ "f1_sample_sd": 0.018643640071557464
586
+ },
587
+ {
588
+ "arm": "structural_copy_control",
589
+ "label": "Doubt",
590
+ "support": 22,
591
+ "precision_mean": 0.4455314009661836,
592
+ "precision_sample_sd": 0.04067072694857813,
593
+ "recall_mean": 1.0,
594
+ "recall_sample_sd": 0.0,
595
+ "f1_mean": 0.615686274509804,
596
+ "f1_sample_sd": 0.039427200694482675
597
+ },
598
+ {
599
+ "arm": "structural_copy_control",
600
+ "label": "Exaggeration,Minimisation",
601
+ "support": 22,
602
+ "precision_mean": 0.4364176150420584,
603
+ "precision_sample_sd": 0.025093958106097894,
604
+ "recall_mean": 0.8787878787878788,
605
+ "recall_sample_sd": 0.05248638810814775,
606
+ "f1_mean": 0.5829803047194351,
607
+ "f1_sample_sd": 0.030899288356345502
608
+ },
609
+ {
610
+ "arm": "structural_copy_control",
611
+ "label": "Flag-Waving",
612
+ "support": 17,
613
+ "precision_mean": 0.3814814814814815,
614
+ "precision_sample_sd": 0.05010277503136551,
615
+ "recall_mean": 0.803921568627451,
616
+ "recall_sample_sd": 0.17970885078258195,
617
+ "f1_mean": 0.5138595287119652,
618
+ "f1_sample_sd": 0.06400931989326974
619
+ },
620
+ {
621
+ "arm": "structural_copy_control",
622
+ "label": "Loaded_Language",
623
+ "support": 45,
624
+ "precision_mean": 0.8492581841638446,
625
+ "precision_sample_sd": 0.016026591848331686,
626
+ "recall_mean": 1.0,
627
+ "recall_sample_sd": 0.0,
628
+ "f1_mean": 0.9184311025253588,
629
+ "f1_sample_sd": 0.009372233868132662
630
+ },
631
+ {
632
+ "arm": "structural_copy_control",
633
+ "label": "Name_Calling,Labeling",
634
+ "support": 33,
635
+ "precision_mean": 0.6375,
636
+ "precision_sample_sd": 0.03410563654946854,
637
+ "recall_mean": 0.9696969696969697,
638
+ "recall_sample_sd": 0.030303030303030276,
639
+ "f1_mean": 0.7685185185185185,
640
+ "f1_sample_sd": 0.02023900779105559
641
+ },
642
+ {
643
+ "arm": "structural_copy_control",
644
+ "label": "Repetition",
645
+ "support": 23,
646
+ "precision_mean": 0.44863933452168747,
647
+ "precision_sample_sd": 0.01520014564765896,
648
+ "recall_mean": 0.9420289855072463,
649
+ "recall_sample_sd": 0.06641414050660639,
650
+ "f1_mean": 0.607248835423033,
651
+ "f1_sample_sd": 0.019858627551810427
652
+ },
653
+ {
654
+ "arm": "structural_copy_control",
655
+ "label": "Slogans",
656
+ "support": 14,
657
+ "precision_mean": 0.3274318274318274,
658
+ "precision_sample_sd": 0.05434598572773684,
659
+ "recall_mean": 0.4523809523809524,
660
+ "recall_sample_sd": 0.1486904285332952,
661
+ "f1_mean": 0.3715828924162258,
662
+ "f1_sample_sd": 0.07106660236343577
663
+ },
664
+ {
665
+ "arm": "structural_copy_control",
666
+ "label": "Thought-terminating_Cliches",
667
+ "support": 7,
668
+ "precision_mean": 0.1142156862745098,
669
+ "precision_sample_sd": 0.012848374923608973,
670
+ "recall_mean": 0.38095238095238093,
671
+ "recall_sample_sd": 0.1649572197684645,
672
+ "f1_mean": 0.17331433998100665,
673
+ "f1_sample_sd": 0.029065880233243847
674
+ },
675
+ {
676
+ "arm": "structural_copy_control",
677
+ "label": "Whataboutism,Straw_Men,Red_Herring",
678
+ "support": 10,
679
+ "precision_mean": 0.19344598054275472,
680
+ "precision_sample_sd": 0.03373872471827226,
681
+ "recall_mean": 0.5666666666666667,
682
+ "recall_sample_sd": 0.2081665999466133,
683
+ "f1_mean": 0.28584083690299267,
684
+ "f1_sample_sd": 0.06078854830133676
685
+ },
686
+ {
687
+ "arm": "existing_specialized_separate_lineage",
688
+ "label": "Appeal_to_Authority",
689
+ "support": 11,
690
+ "precision_mean": 0.37836021505376344,
691
+ "precision_sample_sd": 0.10546365636105155,
692
+ "recall_mean": 0.6666666666666666,
693
+ "recall_sample_sd": 0.2287828616748712,
694
+ "f1_mean": 0.46885361552028215,
695
+ "f1_sample_sd": 0.09502747629890008
696
+ },
697
+ {
698
+ "arm": "existing_specialized_separate_lineage",
699
+ "label": "Appeal_to_fear-prejudice",
700
+ "support": 24,
701
+ "precision_mean": 0.525708635464733,
702
+ "precision_sample_sd": 0.02235198659615819,
703
+ "recall_mean": 0.8888888888888888,
704
+ "recall_sample_sd": 0.048112522432468774,
705
+ "f1_mean": 0.6599065351235072,
706
+ "f1_sample_sd": 0.015362381542320084
707
+ },
708
+ {
709
+ "arm": "existing_specialized_separate_lineage",
710
+ "label": "Bandwagon,Reductio_ad_hitlerum",
711
+ "support": 7,
712
+ "precision_mean": 0.44485144485144484,
713
+ "precision_sample_sd": 0.016487284575115283,
714
+ "recall_mean": 0.7619047619047619,
715
+ "recall_sample_sd": 0.1649572197684645,
716
+ "f1_mean": 0.5571428571428572,
717
+ "f1_sample_sd": 0.051507875363771265
718
+ },
719
+ {
720
+ "arm": "existing_specialized_separate_lineage",
721
+ "label": "Black-and-White_Fallacy",
722
+ "support": 12,
723
+ "precision_mean": 0.4662644544997486,
724
+ "precision_sample_sd": 0.004538274146363688,
725
+ "recall_mean": 0.5833333333333334,
726
+ "recall_sample_sd": 0.08333333333333331,
727
+ "f1_mean": 0.516747552149851,
728
+ "f1_sample_sd": 0.03589484965697345
729
+ },
730
+ {
731
+ "arm": "existing_specialized_separate_lineage",
732
+ "label": "Causal_Oversimplification",
733
+ "support": 18,
734
+ "precision_mean": 0.4472934472934473,
735
+ "precision_sample_sd": 0.013055771210700413,
736
+ "recall_mean": 0.8333333333333333,
737
+ "recall_sample_sd": 0.1469861839480328,
738
+ "f1_mean": 0.5781794553724379,
739
+ "f1_sample_sd": 0.028407563015813122
740
+ },
741
+ {
742
+ "arm": "existing_specialized_separate_lineage",
743
+ "label": "Doubt",
744
+ "support": 22,
745
+ "precision_mean": 0.5045661001171619,
746
+ "precision_sample_sd": 0.023581174969843577,
747
+ "recall_mean": 1.0,
748
+ "recall_sample_sd": 0.0,
749
+ "f1_mean": 0.6704939668174963,
750
+ "f1_sample_sd": 0.020973132363659372
751
+ },
752
+ {
753
+ "arm": "existing_specialized_separate_lineage",
754
+ "label": "Exaggeration,Minimisation",
755
+ "support": 22,
756
+ "precision_mean": 0.44589793281653745,
757
+ "precision_sample_sd": 0.010987847993006481,
758
+ "recall_mean": 0.9393939393939394,
759
+ "recall_sample_sd": 0.06943296507508849,
760
+ "f1_mean": 0.6043956043956044,
761
+ "f1_sample_sd": 0.022305256187015846
762
+ },
763
+ {
764
+ "arm": "existing_specialized_separate_lineage",
765
+ "label": "Flag-Waving",
766
+ "support": 17,
767
+ "precision_mean": 0.5576388888888889,
768
+ "precision_sample_sd": 0.03705367103320583,
769
+ "recall_mean": 0.8823529411764706,
770
+ "recall_sample_sd": 0.11764705882352944,
771
+ "f1_mean": 0.680769868923179,
772
+ "f1_sample_sd": 0.04164658769406446
773
+ },
774
+ {
775
+ "arm": "existing_specialized_separate_lineage",
776
+ "label": "Loaded_Language",
777
+ "support": 45,
778
+ "precision_mean": 0.8387650085763294,
779
+ "precision_sample_sd": 0.01782556577257849,
780
+ "recall_mean": 1.0,
781
+ "recall_sample_sd": 0.0,
782
+ "f1_mean": 0.9122448979591837,
783
+ "f1_sample_sd": 0.01060439269940129
784
+ },
785
+ {
786
+ "arm": "existing_specialized_separate_lineage",
787
+ "label": "Name_Calling,Labeling",
788
+ "support": 33,
789
+ "precision_mean": 0.6957104190221673,
790
+ "precision_sample_sd": 0.019201490809269257,
791
+ "recall_mean": 0.9696969696969697,
792
+ "recall_sample_sd": 0.030303030303030276,
793
+ "f1_mean": 0.8101049442821594,
794
+ "f1_sample_sd": 0.022092733328802537
795
+ },
796
+ {
797
+ "arm": "existing_specialized_separate_lineage",
798
+ "label": "Repetition",
799
+ "support": 23,
800
+ "precision_mean": 0.4781557067271353,
801
+ "precision_sample_sd": 0.009897987322673213,
802
+ "recall_mean": 0.9420289855072463,
803
+ "recall_sample_sd": 0.06641414050660639,
804
+ "f1_mean": 0.6337774426009719,
805
+ "f1_sample_sd": 0.016444118333485254
806
+ },
807
+ {
808
+ "arm": "existing_specialized_separate_lineage",
809
+ "label": "Slogans",
810
+ "support": 14,
811
+ "precision_mean": 0.4334215167548501,
812
+ "precision_sample_sd": 0.05811581861282867,
813
+ "recall_mean": 0.6904761904761905,
814
+ "recall_sample_sd": 0.1649572197684645,
815
+ "f1_mean": 0.5201316298877274,
816
+ "f1_sample_sd": 0.018567913895656114
817
+ },
818
+ {
819
+ "arm": "existing_specialized_separate_lineage",
820
+ "label": "Thought-terminating_Cliches",
821
+ "support": 7,
822
+ "precision_mean": 0.18446115288220552,
823
+ "precision_sample_sd": 0.03641251252864963,
824
+ "recall_mean": 0.6666666666666666,
825
+ "recall_sample_sd": 0.1649572197684645,
826
+ "f1_mean": 0.2868626868626869,
827
+ "f1_sample_sd": 0.0511620864429468
828
+ },
829
+ {
830
+ "arm": "existing_specialized_separate_lineage",
831
+ "label": "Whataboutism,Straw_Men,Red_Herring",
832
+ "support": 10,
833
+ "precision_mean": 0.2777777777777778,
834
+ "precision_sample_sd": 0.0481125224324688,
835
+ "recall_mean": 0.4,
836
+ "recall_sample_sd": 0.0,
837
+ "f1_mean": 0.32634032634032634,
838
+ "f1_sample_sd": 0.03229931575886014
839
+ }
840
+ ],
841
+ "compression_ladder": [
842
+ {
843
+ "name": "24L upstream text teacher",
844
+ "layers": 24,
845
+ "parameters": 752393024,
846
+ "weight_bytes": 1504825632,
847
+ "weight_mib": 1435.1135559082031,
848
+ "weight_sha256": "30783da4651259c26b8a204821ea426b9526431e0c14da0d1c6fa284e7414e5e",
849
+ "kd_stage": "",
850
+ "mean_total_loss": "",
851
+ "mean_interface_loss": "",
852
+ "mean_final_loss": ""
853
+ },
854
+ {
855
+ "name": "8L task-free KD",
856
+ "layers": 8,
857
+ "parameters": 420318144,
858
+ "weight_bytes": 840647880,
859
+ "weight_mib": 801.7042922973633,
860
+ "weight_sha256": "086fbaf9a4838ecde56cd3541c9c3212b22d0c5ee745cf2c1c5e0bb31e0ec22c",
861
+ "kd_stage": "24to8",
862
+ "mean_total_loss": 0.17622653172702485,
863
+ "mean_interface_loss": 0.21562811763578793,
864
+ "mean_final_loss": 0.16626427527853593
865
+ },
866
+ {
867
+ "name": "6L task-free KD",
868
+ "layers": 6,
869
+ "parameters": 377207424,
870
+ "weight_bytes": 754423424,
871
+ "weight_mib": 719.4742431640625,
872
+ "weight_sha256": "492749c20f77a42f6bc993f0927a82217b94f872b7eacf25612f7520c9bc28ce",
873
+ "kd_stage": "8to6",
874
+ "mean_total_loss": 0.03382104352249371,
875
+ "mean_interface_loss": 0.04486069045515251,
876
+ "mean_final_loss": 0.031147060785087888
877
+ },
878
+ {
879
+ "name": "4L task-free KD root",
880
+ "layers": 4,
881
+ "parameters": 334096704,
882
+ "weight_bytes": 668198976,
883
+ "weight_mib": 637.2442016601562,
884
+ "weight_sha256": "2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3",
885
+ "kd_stage": "6to4",
886
+ "mean_total_loss": 0.047020394468859195,
887
+ "mean_interface_loss": 0.07048421185527332,
888
+ "mean_final_loss": 0.043119939065377366
889
+ }
890
+ ]
891
+ }
release_manifest.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "schema": "qwen35-unified-classification-base-release-v2",
3
  "status": "complete",
4
  "repo_id": "mp-juuuns/qwen35-standalone4l-classification-base",
5
  "old_repo_preserved": {
@@ -10,6 +10,7 @@
10
  "root_weight_sha256": "2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3",
11
  "root_multilingual_metadata_scope": "inherited tokenizer/architecture coverage; no direct multilingual downstream evaluation",
12
  "benchmark_schema": "standalone4l-complete-transfer-benchmark-v2",
 
13
  "nested_specialized_is_separate_lineage": true,
14
  "source_data_redistributed": false,
15
  "files_before_manifest": [
@@ -30,8 +31,8 @@
30
  {
31
  "path": "README.md",
32
  "role": "root model card",
33
- "bytes": 12909,
34
- "sha256": "5fcf007de6b6e828711c874310de1e042f1dada6989c70930ac0881285a9dc61",
35
  "source": "docs/huggingface/UNIFIED_BASE_RELEASE_MODEL_CARD.md"
36
  },
37
  {
@@ -55,6 +56,111 @@
55
  "sha256": "2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b",
56
  "source": "generated"
57
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
  {
59
  "path": "benchmark/per_label_metrics.csv",
60
  "role": "per-label metrics by arm and seed",
 
1
  {
2
+ "schema": "qwen35-unified-classification-base-release-v3",
3
  "status": "complete",
4
  "repo_id": "mp-juuuns/qwen35-standalone4l-classification-base",
5
  "old_repo_preserved": {
 
10
  "root_weight_sha256": "2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3",
11
  "root_multilingual_metadata_scope": "inherited tokenizer/architecture coverage; no direct multilingual downstream evaluation",
12
  "benchmark_schema": "standalone4l-complete-transfer-benchmark-v2",
13
+ "figure_schema": "standalone4l-matplotlib-figures-v1",
14
  "nested_specialized_is_separate_lineage": true,
15
  "source_data_redistributed": false,
16
  "files_before_manifest": [
 
31
  {
32
  "path": "README.md",
33
  "role": "root model card",
34
+ "bytes": 14017,
35
+ "sha256": "49cb7082e65f26835c90131ce7ff05f47ebd77ec9352dad491a0083ede70bd50",
36
  "source": "docs/huggingface/UNIFIED_BASE_RELEASE_MODEL_CARD.md"
37
  },
38
  {
 
56
  "sha256": "2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b",
57
  "source": "generated"
58
  },
59
+ {
60
+ "path": "benchmark/figures/01_overall_transfer_quality.csv",
61
+ "role": "Matplotlib benchmark figures and source data",
62
+ "bytes": 500,
63
+ "sha256": "769a516e53417e16b61cf30d95a02f6c812c0b1536242cdbf836da779753efbd",
64
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/01_overall_transfer_quality.csv"
65
+ },
66
+ {
67
+ "path": "benchmark/figures/01_overall_transfer_quality.pdf",
68
+ "role": "Matplotlib benchmark figures and source data",
69
+ "bytes": 18071,
70
+ "sha256": "7221e0d2feb356e4d7f44aa0381e576254f734837e3f54fb0dfb98795ec19ded",
71
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/01_overall_transfer_quality.pdf"
72
+ },
73
+ {
74
+ "path": "benchmark/figures/01_overall_transfer_quality.png",
75
+ "role": "Matplotlib benchmark figures and source data",
76
+ "bytes": 82127,
77
+ "sha256": "6d35b5964791cfff2701e43831996a9aad7d607305ab3031b0782bed2ffffa66",
78
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/01_overall_transfer_quality.png"
79
+ },
80
+ {
81
+ "path": "benchmark/figures/02_per_label_transfer_metrics.csv",
82
+ "role": "Matplotlib benchmark figures and source data",
83
+ "bytes": 1169,
84
+ "sha256": "50a3ceb0e3146ba8cade9c86c4b59c7db6cbedfc41fac39a811d7f39f4bdeb0d",
85
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/02_per_label_transfer_metrics.csv"
86
+ },
87
+ {
88
+ "path": "benchmark/figures/02_per_label_transfer_metrics.pdf",
89
+ "role": "Matplotlib benchmark figures and source data",
90
+ "bytes": 21379,
91
+ "sha256": "10756816e8ad800f5d872907986680abf637e76fe47f9790f27645f8f231ae4a",
92
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/02_per_label_transfer_metrics.pdf"
93
+ },
94
+ {
95
+ "path": "benchmark/figures/02_per_label_transfer_metrics.png",
96
+ "role": "Matplotlib benchmark figures and source data",
97
+ "bytes": 138298,
98
+ "sha256": "5b558738b405a7b04bb30f38b6e53af21b2392ea12cad0d777e9d6d6985d3d04",
99
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/02_per_label_transfer_metrics.png"
100
+ },
101
+ {
102
+ "path": "benchmark/figures/03_quality_efficiency.csv",
103
+ "role": "Matplotlib benchmark figures and source data",
104
+ "bytes": 448,
105
+ "sha256": "f2a34cbcd700b2663cf761a5fb1dbd3aa99f310a96d4b4477a3a4529bee1a1b3",
106
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/03_quality_efficiency.csv"
107
+ },
108
+ {
109
+ "path": "benchmark/figures/03_quality_efficiency.pdf",
110
+ "role": "Matplotlib benchmark figures and source data",
111
+ "bytes": 21933,
112
+ "sha256": "18c81ce50fc7871b000c9e387f0f7d7780002cb2bd249153aadfa55a186342c0",
113
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/03_quality_efficiency.pdf"
114
+ },
115
+ {
116
+ "path": "benchmark/figures/03_quality_efficiency.png",
117
+ "role": "Matplotlib benchmark figures and source data",
118
+ "bytes": 99374,
119
+ "sha256": "08ed7ca8193d5f9065645fb22491ab88ee260d43566e9013aa0f98d7ab4dde87",
120
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/03_quality_efficiency.png"
121
+ },
122
+ {
123
+ "path": "benchmark/figures/04_compression_ladder.csv",
124
+ "role": "Matplotlib benchmark figures and source data",
125
+ "bytes": 352,
126
+ "sha256": "2dfac2b8330724ce46a22d8f3ddfeffc767f52b93a93cb1a833c89c5820ffccc",
127
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/04_compression_ladder.csv"
128
+ },
129
+ {
130
+ "path": "benchmark/figures/04_compression_ladder.pdf",
131
+ "role": "Matplotlib benchmark figures and source data",
132
+ "bytes": 19490,
133
+ "sha256": "953d6e356498127b2a607530fac2e41b07b4bf38ed0af945f778b784dc6f7b23",
134
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/04_compression_ladder.pdf"
135
+ },
136
+ {
137
+ "path": "benchmark/figures/04_compression_ladder.png",
138
+ "role": "Matplotlib benchmark figures and source data",
139
+ "bytes": 120809,
140
+ "sha256": "9f6c1d6fdefd45711fcfa57a7d0d91f6baaa3a5ee0b44a36554a9c74febe7090",
141
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/04_compression_ladder.png"
142
+ },
143
+ {
144
+ "path": "benchmark/figures/README.md",
145
+ "role": "Matplotlib benchmark figures and source data",
146
+ "bytes": 683,
147
+ "sha256": "7e00107e6576da117689dca0c164c497ce32e9d93c034331e2e161625afc2092",
148
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/README.md"
149
+ },
150
+ {
151
+ "path": "benchmark/figures/manifest.json",
152
+ "role": "Matplotlib benchmark figures and source data",
153
+ "bytes": 2697,
154
+ "sha256": "b3e3da2fcf1ae8ea9c88019c289e3d45e7945df655218d5c0d3b905769811f50",
155
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/manifest.json"
156
+ },
157
+ {
158
+ "path": "benchmark/figures/source_summary.json",
159
+ "role": "Matplotlib benchmark figures and source data",
160
+ "bytes": 26913,
161
+ "sha256": "3d2c9fcf9c6cc7da1a885968b70a42eaa38659d317c9d3290a2b3066f3f9ee6a",
162
+ "source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/source_summary.json"
163
+ },
164
  {
165
  "path": "benchmark/per_label_metrics.csv",
166
  "role": "per-label metrics by arm and seed",