mp-juuuns commited on
Commit
49f5d40
·
verified ·
1 Parent(s): 9fc8c19

Add complete frozen benchmark metrics and bounded multilingual metadata

Browse files
.gitattributes CHANGED
@@ -1,2 +1,4 @@
1
  *.safetensors filter=lfs diff=lfs merge=lfs -text
2
- tokenizer.json filter=lfs diff=lfs merge=lfs -text
 
 
 
1
  *.safetensors filter=lfs diff=lfs merge=lfs -text
2
+ models/semeval-propaganda/seeds/seed42/tokenizer.json filter=lfs diff=lfs merge=lfs -text
3
+ models/semeval-propaganda/seeds/seed43/tokenizer.json filter=lfs diff=lfs merge=lfs -text
4
+ models/semeval-propaganda/tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -4,10 +4,12 @@ base_model: Qwen/Qwen3.5-0.8B
4
  pipeline_tag: feature-extraction
5
  library_name: transformers
6
  language:
7
- - en
8
  tags:
 
9
  - qwen3.5
10
  - classification-backbone
 
11
  - knowledge-distillation
12
  - model-compression
13
  - edge-ai
@@ -15,18 +17,34 @@ tags:
15
 
16
  # Qwen3.5 Standalone 4L Classification Base
17
 
18
- This repository publishes a **headless, classification-ready four-layer
19
- Qwen3.5 text backbone**. The root model has no task labels and no classification
20
- head. It is intended as a compact starting point for single-label or multilabel
21
- classifiers, not as a chat model or a ready-made universal classifier.
 
22
 
23
  The repository follows one integrated 1-1-1 layout:
24
 
25
- - **Root model:** task-agnostic four-layer base trained on unlabeled general
26
- text.
27
- - **`models/semeval-propaganda/`:** the existing SemEval-derived propaganda
28
- classifier as a separate task-specific lineage.
29
- - **`distillation/`:** a reusable 24L→8L→6L→4L platform for your own data.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
 
31
  ## Load the headless base
32
 
@@ -38,10 +56,8 @@ tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=False)
38
  backbone = AutoModel.from_pretrained(model_id, trust_remote_code=False)
39
  ```
40
 
41
- The root uses `Qwen3_5TextModel` in Transformers 5.13.0. It has four text
42
- layers in the sequence `linear full linear → full`, hidden size 1,024, and
43
- the full upstream Qwen3.5 tokenizer. The published weight file contains no
44
- `score.weight`.
45
 
46
  ## Make your own classifier
47
 
@@ -58,72 +74,199 @@ qwen35-distill finetune \
58
  --output my-classifier-4l
59
  ```
60
 
61
- For a full task-specific compression run, first fine-tune a 24-layer teacher,
62
- then repeat `materialize-classifier` and `distill-classifier` for `24to8`,
63
- `8to6`, and `6to4`. Exact JSONL formats and commands are in
64
  [`distillation/README.md`](distillation/README.md).
65
 
66
  ## How the root weights were made
67
 
68
- The text backbone and full tokenizer were extracted from
69
- [`Qwen/Qwen3.5-0.8B`](https://huggingface.co/Qwen/Qwen3.5-0.8B). Students were
70
- trained sequentially through 24→8→6→4 using hidden-boundary and final
71
- representation matching on unlabeled text.
72
 
73
  The task-free training set was exactly 4,096 deterministically shuffled,
74
  non-empty rows from the first locally cached WikiText-103 raw training shard.
75
- No SemEval examples, labels, logits, thresholds, or evidence/span objectives
76
- were read while training these root weights.
77
 
78
- WikiText is not redistributed here. Its current Hugging Face page has a license
79
- wording discrepancy: metadata lists CC BY-SA 3.0 and GFDL, while the prose says
80
- CC BY-SA 4.0. Users should review the
81
- [`Salesforce/wikitext`](https://huggingface.co/datasets/Salesforce/wikitext)
82
- terms directly.
83
 
84
- ## One-task transfer check
 
 
 
85
 
86
- We attached a fresh 14-label head and used the frozen SemEval-derived
87
- article-level protocol only as a transfer probe.
88
 
89
- | Initialization | Opened-test Macro-F1, mean ± sample SD |
90
- |---|---:|
91
- | Task-agnostic 4L base | **0.58748 ± 0.01579** |
92
- | Same 4L structural copy without task-free KD | 0.51809 ± 0.01631 |
93
- | Existing separate specialized 4L lineage | 0.58757 ± 0.00717 |
94
-
95
- Results use seeds 41/42/43 and a previously opened 55-article public test split.
96
- They are exploratory, not untouched confirmatory evidence. The derived task
97
- measures article-level presence of 14 techniques and is not identical to the
98
- official SemEval span- or fragment-level task. Three seed values are not a
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
99
  confidence interval.
100
 
101
- This check supports transfer on one English classification task only. It does
102
- not establish universal classification quality, multilingual generalization,
103
- calibration, safety, or production suitability.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
104
 
105
  ## Separate SemEval model
106
 
107
- The ready-to-use propaganda model is under
108
  [`models/semeval-propaganda/`](models/semeval-propaganda/). It preserves the
109
- currently published seed-41/42/43 checkpoints and links to the original
110
- repository at immutable revision
111
  [`e7e0ff1`](https://huggingface.co/mp-juuuns/qwen35-standalone4l-propaganda-classifier/tree/e7e0ff16828052687d2e8dd7849e7a521629cf38).
112
 
113
- That specialized model is **not** claimed to descend from this new task-free
114
- base. It has its own SemEval task-specific shrink/distillation and fine-tuning
115
- history and a reduced 128k vocabulary, while the root base retains the full
116
- upstream tokenizer.
 
117
 
118
  ## Limitations
119
 
120
  - A classification head and labeled training are required before root-model
121
  predictions have task meaning.
122
- - The task-free corpus is small: 4,096 rows from one WikiText shard.
123
- - Only one downstream transfer task was measured.
 
 
 
 
 
 
 
124
  - Default layer maps are documented structural choices, not universal optima.
125
- - Long-context, multilingual, calibration, robustness, fairness, and device
126
- performance of the new root base have not been established.
127
  - Do not use either model as a fact checker, safety oracle, or autonomous
128
  decision maker.
129
 
@@ -134,7 +277,7 @@ and data terms. Qwen3.5 is attributed to Qwen. Transformers, PyTorch,
134
  Hugging Face Hub, WikiText, and SemEval remain the work of their respective
135
  authors. No WikiText or SemEval source records are redistributed.
136
 
137
- Machine-readable training manifests, transfer summaries, hashes, and the
138
- release inventory are included under `provenance/`, `benchmark/`, and
139
- `release_manifest.json`.
140
-
 
4
  pipeline_tag: feature-extraction
5
  library_name: transformers
6
  language:
7
+ - multilingual
8
  tags:
9
+ - multilingual
10
  - qwen3.5
11
  - classification-backbone
12
+ - text-classification
13
  - knowledge-distillation
14
  - model-compression
15
  - edge-ai
 
17
 
18
  # Qwen3.5 Standalone 4L Classification Base
19
 
20
+ This repository publishes a **headless, multilingual-input, classification-ready
21
+ four-layer Qwen3.5 text backbone**. The root model has no task labels and no
22
+ classification head. It is a compact starting point for user-trained
23
+ single-label or multilabel classifiers, not a chat model or a ready-made
24
+ universal classifier.
25
 
26
  The repository follows one integrated 1-1-1 layout:
27
 
28
+ 1. **Repository root:** task-agnostic four-layer base trained on unlabeled
29
+ general text.
30
+ 2. **`models/semeval-propaganda/`:** ready-to-use SemEval-derived propaganda
31
+ classifier preserved as a separate task-specific lineage.
32
+ 3. **`distillation/`:** reusable 24L→8L→6L→4L platform for user-owned data.
33
+
34
+ ## Model at a glance
35
+
36
+ | Item | Root model |
37
+ |---|---|
38
+ | Upstream | [`Qwen/Qwen3.5-0.8B`](https://huggingface.co/Qwen/Qwen3.5-0.8B) |
39
+ | Runtime class | `Qwen3_5TextModel` in Transformers 5.13.0 |
40
+ | Text layers | 4: `linear_attention → full_attention → linear_attention → full_attention` |
41
+ | Hidden / FFN | 1,024 / 3,584 |
42
+ | Vocabulary | full upstream Qwen3.5 tokenizer, 248,320 entries |
43
+ | Root parameters | 334,096,704 |
44
+ | Root BF16 weights | 668,198,976 bytes (637.24 MiB) |
45
+ | Output | hidden states; no LM head, task head, labels, or thresholds |
46
+ | Task-free KD data | 4,096 unlabeled WikiText-103 raw rows |
47
+ | Downstream evidence | one English 14-label SemEval-derived transfer probe |
48
 
49
  ## Load the headless base
50
 
 
56
  backbone = AutoModel.from_pretrained(model_id, trust_remote_code=False)
57
  ```
58
 
59
+ The published root weight file contains no `score.weight`. A fresh task head and
60
+ labeled fine-tuning are required before outputs have class meaning.
 
 
61
 
62
  ## Make your own classifier
63
 
 
74
  --output my-classifier-4l
75
  ```
76
 
77
+ For a full task-specific compression run, fine-tune a 24-layer teacher, then
78
+ repeat `materialize-classifier` and `distill-classifier` for `24to8`, `8to6`,
79
+ and `6to4`. Exact JSONL formats, split rules, manifests, and commands are in
80
  [`distillation/README.md`](distillation/README.md).
81
 
82
  ## How the root weights were made
83
 
84
+ The text backbone and full tokenizer were extracted from the upstream
85
+ Qwen3.5-0.8B model. Students were trained sequentially through 24→8→6→4 using
86
+ hidden-boundary and final-representation matching on unlabeled text.
 
87
 
88
  The task-free training set was exactly 4,096 deterministically shuffled,
89
  non-empty rows from the first locally cached WikiText-103 raw training shard.
90
+ No SemEval examples, labels, logits, thresholds, classification heads, or
91
+ evidence/span objectives were read while training the root weights.
92
 
93
+ | KD stage | Selected source layers | Mean total loss | Mean interface loss | Mean final loss |
94
+ |---|---|---:|---:|---:|
95
+ | 24→8 | `0,4,6,11,13,16,20,23` | 0.176227 | 0.215628 | 0.166264 |
96
+ | 8→6 | `0,1,3,4,6,7` | 0.033821 | 0.044861 | 0.031147 |
97
+ | 6→4 | `0,2,3,5` | 0.047020 | 0.070484 | 0.043120 |
98
 
99
+ These are training-objective values, not downstream quality scores. Exact
100
+ parameter counts, BF16 file sizes, hashes, and stage values are in
101
+ [`benchmark/compression_ladder.csv`](benchmark/compression_ladder.csv) and
102
+ [`provenance/task_agnostic_stages/`](provenance/task_agnostic_stages/).
103
 
104
+ ## Downstream benchmark contract
 
105
 
106
+ Because the root is headless, it cannot be scored as a classifier without
107
+ adaptation. Its task-appropriate benchmark is therefore a **controlled transfer
108
+ probe**:
109
+
110
+ - attach a fresh randomly initialized 14-label classification head;
111
+ - train all parameters for five epochs on the same SemEval-derived train split;
112
+ - select one global threshold on the epoch-5 calibration split;
113
+ - evaluate once per seed on the already-opened 55-article test split;
114
+ - compare against the same four-layer structural copy without task-free KD.
115
+
116
+ This derived task measures article-level presence of 14 propaganda techniques.
117
+ It is not the official SemEval span- or fragment-level task. Results are
118
+ exploratory because the public test split had already been opened.
119
+
120
+ ### Frozen protocol
121
+
122
+ | Setting | Value |
123
+ |---|---|
124
+ | Seeds | 41, 42, 43 |
125
+ | Train / calibration / test articles | 260 / 56 / 55 |
126
+ | Test windows | 434 |
127
+ | Epochs | 5, no early stopping |
128
+ | Input windows | max length 256, overlap stride 128 |
129
+ | Article aggregation | label-wise maximum probability |
130
+ | Batch | micro 1, gradient accumulation 32, effective 32 |
131
+ | Optimizer | AdamW, LR 2e-5, weight decay 0.01, gradient clip 1.0 |
132
+ | Precision / device | BF16 / NVIDIA GeForce RTX 5070 Ti |
133
+ | Threshold selection | epoch-5 calibration Macro-F1, then Micro-F1 |
134
+
135
+ ### Overall results by seed
136
+
137
+ | Initialization | Seed | Threshold | Macro-F1 | Micro-F1 | Exact match | Predicted positive rate |
138
+ |---|---:|---:|---:|---:|---:|---:|
139
+ | Task-agnostic KD base | 41 | 0.15 | 0.58831 | 0.64681 | 0.05455 | 0.57143 |
140
+ | Task-agnostic KD base | 42 | 0.30 | 0.60284 | 0.66552 | 0.03636 | 0.40909 |
141
+ | Task-agnostic KD base | 43 | 0.10 | 0.57128 | 0.64653 | 0.00000 | 0.51558 |
142
+ | Structural copy, no task-free KD | 41 | 0.05 | 0.51694 | 0.57740 | 0.00000 | 0.71299 |
143
+ | Structural copy, no task-free KD | 42 | 0.20 | 0.53494 | 0.60834 | 0.00000 | 0.62078 |
144
+ | Structural copy, no task-free KD | 43 | 0.20 | 0.50238 | 0.58564 | 0.00000 | 0.59610 |
145
+
146
+ ### Three-seed summary
147
+
148
+ Values are mean ± sample SD across the three training seeds. The SD is not a
149
  confidence interval.
150
 
151
+ | Initialization | Macro-F1 | Micro-F1 | Exact match | Predicted positive rate |
152
+ |---|---:|---:|---:|---:|
153
+ | **Task-agnostic KD base** | **0.58748 ± 0.01579** | **0.65295 ± 0.01088** | **0.03030 ± 0.02777** | 0.49870 ± 0.08248 |
154
+ | Structural copy, no task-free KD | 0.51809 ± 0.01631 | 0.59046 ± 0.01603 | 0.00000 ± 0.00000 | 0.64329 ± 0.06161 |
155
+ | Existing specialized 4L, separate lineage | 0.58757 ± 0.00717 | see nested reports | see nested reports | see nested reports |
156
+
157
+ The task-agnostic KD initialization improved same-seed mean Macro-F1 by
158
+ `+0.06939` over the structural-copy control. This is an observed comparison
159
+ under this protocol, not a significance or universal-superiority claim.
160
+
161
+ ### Task-agnostic base: per-label test results
162
+
163
+ Precision, recall, and F1 are three-seed means. Support is the fixed number of
164
+ positive test articles, not a three-seed sum.
165
+
166
+ | Label | Support | Precision | Recall | F1 |
167
+ |---|---:|---:|---:|---:|
168
+ | `Appeal_to_Authority` | 11 | 0.3959 | 0.5758 | 0.4657 |
169
+ | `Appeal_to_fear-prejudice` | 24 | 0.5540 | 0.7361 | 0.6278 |
170
+ | `Bandwagon,Reductio_ad_hitlerum` | 7 | 0.4852 | 0.8095 | 0.6007 |
171
+ | `Black-and-White_Fallacy` | 12 | 0.3668 | 0.4722 | 0.4054 |
172
+ | `Causal_Oversimplification` | 18 | 0.5429 | 0.7963 | 0.6447 |
173
+ | `Doubt` | 22 | 0.5224 | 0.9848 | 0.6802 |
174
+ | `Exaggeration,Minimisation` | 22 | 0.4621 | 0.7727 | 0.5727 |
175
+ | `Flag-Waving` | 17 | 0.5643 | 0.8235 | 0.6465 |
176
+ | `Loaded_Language` | 45 | 0.8649 | 0.9852 | 0.9209 |
177
+ | `Name_Calling,Labeling` | 33 | 0.7381 | 0.9091 | 0.8125 |
178
+ | `Repetition` | 23 | 0.5197 | 0.7536 | 0.6148 |
179
+ | `Slogans` | 14 | 0.5161 | 0.7381 | 0.5835 |
180
+ | `Thought-terminating_Cliches` | 7 | 0.3492 | 0.3333 | 0.3190 |
181
+ | `Whataboutism,Straw_Men,Red_Herring` | 10 | 0.2762 | 0.5000 | 0.3304 |
182
+
183
+ Per-seed values and sample SDs for all labels and all arms are available in
184
+ [`benchmark/per_label_metrics.csv`](benchmark/per_label_metrics.csv) and
185
+ [`benchmark/per_label_summary.csv`](benchmark/per_label_summary.csv).
186
+
187
+ ### Efficiency and resource measurements
188
+
189
+ These numbers belong to the fresh-head transfer checkpoints, not a standalone
190
+ headless-root latency benchmark.
191
+
192
+ | Measure | Task-agnostic transfer model |
193
+ |---|---:|
194
+ | Parameters with 14-label head | 334,111,040 |
195
+ | Test inference | 0.08131 ± 0.00243 s/article |
196
+ | Test batch | 55 articles / 434 windows |
197
+ | Peak CUDA allocated | 3,380,975,616 bytes (3.15 GiB) |
198
+ | Total train-and-evaluate wall time | 1,591.69 s mean (26.53 min) |
199
+
200
+ The timing is specific to the recorded RTX 5070 Ti, BF16, batch-1,
201
+ sliding-window protocol and includes article aggregation. It is not a claim
202
+ about CPU, mobile, UNO Q, or other deployment performance. Full seed-level
203
+ resource fields are in
204
+ [`benchmark/resource_metrics.csv`](benchmark/resource_metrics.csv).
205
+
206
+ ### Machine-readable benchmark bundle
207
+
208
+ - [`benchmark/BENCHMARK_CARD.md`](benchmark/BENCHMARK_CARD.md): generated
209
+ benchmark report and interpretation
210
+ - [`benchmark/summary.json`](benchmark/summary.json): complete aggregate
211
+ - [`benchmark/seed_metrics.csv`](benchmark/seed_metrics.csv): overall metrics
212
+ per arm and seed
213
+ - [`benchmark/arm_summary.csv`](benchmark/arm_summary.csv): three-seed means
214
+ and sample SDs
215
+ - [`benchmark/per_label_metrics.csv`](benchmark/per_label_metrics.csv):
216
+ per-label values per arm and seed
217
+ - [`benchmark/per_label_summary.csv`](benchmark/per_label_summary.csv):
218
+ per-label means and sample SDs
219
+ - [`benchmark/resource_metrics.csv`](benchmark/resource_metrics.csv):
220
+ parameters, CUDA memory, wall time, and test timing
221
+ - [`benchmark/reports/`](benchmark/reports/): normalized full frozen reports,
222
+ including article-level probability vectors
223
+
224
+ ## Multilingual scope
225
+
226
+ The root retains the upstream Qwen3.5 tokenizer and multilingual architecture.
227
+ The [official Qwen3.5-0.8B model card](https://huggingface.co/Qwen/Qwen3.5-0.8B)
228
+ states expanded support for **201 languages and dialects** and reports
229
+ multilingual upstream benchmarks such as MMMLU, MMLU-ProX, NOVA-63, INCLUDE,
230
+ Global PIQA, PolyMATH, WMT24++, and MAXIFE. This is why this repository carries
231
+ the Hugging Face `multilingual` language and model tag.
232
+
233
+ However, this four-layer root was distilled on English WikiText and its only
234
+ downstream transfer probe is English SemEval-derived data. **Tokenizer coverage
235
+ and inherited architecture support do not establish retained classification
236
+ quality in all 201 languages.** No multilingual downstream score is reported
237
+ for this root release. Users should fine-tune and evaluate on each intended
238
+ language and domain before making performance claims.
239
 
240
  ## Separate SemEval model
241
 
242
+ The ready-to-use propaganda classifier is under
243
  [`models/semeval-propaganda/`](models/semeval-propaganda/). It preserves the
244
+ published seed-41/42/43 checkpoints and links to the original repository at
245
+ immutable revision
246
  [`e7e0ff1`](https://huggingface.co/mp-juuuns/qwen35-standalone4l-propaganda-classifier/tree/e7e0ff16828052687d2e8dd7849e7a521629cf38).
247
 
248
+ That specialized model is **not** claimed to descend from this task-free base.
249
+ It has its own SemEval task-specific shrink/distillation and fine-tuning history
250
+ and a reduced 128k vocabulary, while the root retains the full upstream
251
+ tokenizer. Its complete reports remain under
252
+ [`models/semeval-propaganda/benchmark/`](models/semeval-propaganda/benchmark/).
253
 
254
  ## Limitations
255
 
256
  - A classification head and labeled training are required before root-model
257
  predictions have task meaning.
258
+ - Task-free KD used only 4,096 rows from one English WikiText shard.
259
+ - Only one downstream task and one hardware/software setting were measured.
260
+ - The 55-article test split was previously opened; results are exploratory.
261
+ - Per-label support ranges from 7 to 45 articles and rare-label estimates are
262
+ unstable.
263
+ - Three seeds describe run-to-run variation; they are not a confidence
264
+ interval.
265
+ - Multilingual input support has not been validated as multilingual downstream
266
+ classification quality for this four-layer root.
267
  - Default layer maps are documented structural choices, not universal optima.
268
+ - Long-context, calibration, robustness, fairness, and production safety have
269
+ not been established.
270
  - Do not use either model as a fact checker, safety oracle, or autonomous
271
  decision maker.
272
 
 
277
  Hugging Face Hub, WikiText, and SemEval remain the work of their respective
278
  authors. No WikiText or SemEval source records are redistributed.
279
 
280
+ The WikiText page currently has a license wording discrepancy: metadata lists
281
+ CC BY-SA 3.0 and GFDL, while prose says CC BY-SA 4.0. Users should inspect
282
+ [`Salesforce/wikitext`](https://huggingface.co/datasets/Salesforce/wikitext)
283
+ directly.
SHA256SUMS CHANGED
@@ -1,13 +1,24 @@
1
  ee05350c7fda0af4da116020dd7b976d731de3beed4f5e717d5bcc95ead24550 .gitattributes
2
  77fd4710def9ec3c0f6225800e0235f15a425abd4a8b03559127fcd782612049 LICENSE
3
- 4f5af67b76f19fca88e242298fa501aa6aa811b4f1f51b481d17abb46e0d39cf README.md
4
- 816c3249d3ec281ef73d34dc3ab7576a48415d250c32977eb94ca37f73b0b810 benchmark/reports/structural_control-seed41.json
5
- 0b19eabe1e31d0012c754db3b31173f8a6fcdee30973f1a1bebd3a370d11064c benchmark/reports/structural_control-seed42.json
6
- 6dde34173462eac50141ff082c18194f3233f8c06853532d9b12756562e69e2e benchmark/reports/structural_control-seed43.json
7
- 6b3a2e0372a83e1e71c2eca9596efc8cd1a3e39c28e25e3373f195431b5858a4 benchmark/reports/task_agnostic-seed41.json
8
- 42dc788983716be0e9847aad87ff79c7e7297884f2635a8d098639be77326639 benchmark/reports/task_agnostic-seed42.json
9
- 7ee52dc94a152a2f767dcff171f3c9cc1072fabd3ebc7b14db53617b7b1d33d5 benchmark/reports/task_agnostic-seed43.json
10
- 18c8d840dd2290ca25c517d52cbcf1ad6c6ae8d9c32c545236e96bd5ada7dab2 benchmark/semeval-transfer-summary.json
 
 
 
 
 
 
 
 
 
 
 
11
  04b007131663760bf3e581e5a953be77044014e87efe1d2a6ca4b72ec0eac978 chat_template.jinja
12
  bb33f3a52d787e15db7d516c6de36a7ddd2c675e7964d5b9309131c4e5d365f0 config.json
13
  f5b7248f7781412e8b5d8f2fb3b0c309568e4e75baf55029b26a6fa985d9d608 distillation/README.md
@@ -25,11 +36,11 @@ a9c49af0ee14b307a08c096bcd1ae55fa9bc67bdb807fa9383fc313246e7edd7 distillation/q
25
  d9b135cf5b0adf4bbfc321200395fbd6112e49fd1a027b30262bcff1c4c0b654 distillation/qwen35_distill/losses.py
26
  00731ae7838317105f9fff960e3da210b2a55954feb37b3ac793bcfc81fdab2d distillation/qwen35_distill/schema.py
27
  4dc432acb70e03848d2e5964e8550065490fda3857aac32fe3b79895ef72026f distillation/qwen35_distill/training.py
28
- 5a1d9e939cd0f350770bcc2a116bc427bb9971bc4fbbbda17ce568e9127185ef docs/RELEASE_CONTRACT.md
29
  a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d merges.txt
30
  2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3 model.safetensors
31
  77fd4710def9ec3c0f6225800e0235f15a425abd4a8b03559127fcd782612049 models/semeval-propaganda/LICENSE
32
- 8dbc2e641280c471b78064808ced7fffcb81c639784a5936b57175975a718f6f models/semeval-propaganda/README.md
33
  0bae347bb54f8089a5fa5450f8366b5c56eb3e598beaf9a87f4385363a402bcb models/semeval-propaganda/benchmark/final4l_aggregate.json
34
  c1bd379680abcf50b47b7d23e4aafbc8b5e7f40a0b0e9700bde21a8ce49af803 models/semeval-propaganda/benchmark/seed41.json
35
  a30e38433d008db4aa7e5fbfd477ab80a0daf50e970f8e7b04397efc1a47843a models/semeval-propaganda/benchmark/seed42.json
@@ -59,7 +70,7 @@ ec734855870758cbfef99031dac2ded85b54c496532e5c514a9a4f63287c3493 provenance/bas
59
  b9ce99b993c4f4563bc2521fc731bfd82f5e8149c141133599fbd8f982ba43d9 provenance/task_agnostic_stages/24to8.json
60
  ec734855870758cbfef99031dac2ded85b54c496532e5c514a9a4f63287c3493 provenance/task_agnostic_stages/6to4.json
61
  65e5383afbbcfe0b7dd78a5411c319ed6bf2586659fb1892aca9ecbd5dfd4993 provenance/task_agnostic_stages/8to6.json
62
- 143945a19235351e00048a99c74799e362eefec3b5e3f3a2ad4c8b64611034a4 release_manifest.json
63
  7b48e7811e56e552fc4aef44189e1d7c9b1352c85d2b3f10ff0bdd1ea5cdd824 requirements.txt
64
  06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523 tokenizer.json
65
  5ab9bed0a4d27949672f65ba1141d6dd5b0514fb9091d3426506ecebb5d5e294 tokenizer_config.json
 
1
  ee05350c7fda0af4da116020dd7b976d731de3beed4f5e717d5bcc95ead24550 .gitattributes
2
  77fd4710def9ec3c0f6225800e0235f15a425abd4a8b03559127fcd782612049 LICENSE
3
+ 5fcf007de6b6e828711c874310de1e042f1dada6989c70930ac0881285a9dc61 README.md
4
+ bf4073de38c3905effd2dcce6f38894f325c8a0a79bedfb2779f0545c86b9c9f benchmark/BENCHMARK_CARD.md
5
+ e01bd7ac59a3212e44b77e5fcfbc7c41ec91d74c998306d2438209d6ef1a3aa7 benchmark/arm_summary.csv
6
+ 2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b benchmark/compression_ladder.csv
7
+ ffc21be32b2106c7bf1df3854097a975ac370da01ac1b732eb5c2ba9075ddeda benchmark/per_label_metrics.csv
8
+ c8e3876abf6cfe960c9332d3bc9d1fbde2f3213f3111351acf83b21bbb184049 benchmark/per_label_summary.csv
9
+ c1bd379680abcf50b47b7d23e4aafbc8b5e7f40a0b0e9700bde21a8ce49af803 benchmark/reports/existing_specialized_separate_lineage-seed41.json
10
+ a30e38433d008db4aa7e5fbfd477ab80a0daf50e970f8e7b04397efc1a47843a benchmark/reports/existing_specialized_separate_lineage-seed42.json
11
+ 96466290d09d27b8bc044a9f9a95765ce12b4d7b02dfb0123e436e7535623353 benchmark/reports/existing_specialized_separate_lineage-seed43.json
12
+ 816c3249d3ec281ef73d34dc3ab7576a48415d250c32977eb94ca37f73b0b810 benchmark/reports/structural_copy_control-seed41.json
13
+ 0b19eabe1e31d0012c754db3b31173f8a6fcdee30973f1a1bebd3a370d11064c benchmark/reports/structural_copy_control-seed42.json
14
+ 6dde34173462eac50141ff082c18194f3233f8c06853532d9b12756562e69e2e benchmark/reports/structural_copy_control-seed43.json
15
+ 6b3a2e0372a83e1e71c2eca9596efc8cd1a3e39c28e25e3373f195431b5858a4 benchmark/reports/task_agnostic_base-seed41.json
16
+ 42dc788983716be0e9847aad87ff79c7e7297884f2635a8d098639be77326639 benchmark/reports/task_agnostic_base-seed42.json
17
+ 7ee52dc94a152a2f767dcff171f3c9cc1072fabd3ebc7b14db53617b7b1d33d5 benchmark/reports/task_agnostic_base-seed43.json
18
+ bf5cefdec3b2f3646114bd18bbf8e5e39721bde768ba968d73f22e96fc9ca8cf benchmark/resource_metrics.csv
19
+ e0b08a958d25243158a03dff76ca153d2ded76cc3f1771e4046a0068688ad6a8 benchmark/seed_metrics.csv
20
+ 3d2c9fcf9c6cc7da1a885968b70a42eaa38659d317c9d3290a2b3066f3f9ee6a benchmark/semeval-transfer-summary.json
21
+ 3d2c9fcf9c6cc7da1a885968b70a42eaa38659d317c9d3290a2b3066f3f9ee6a benchmark/summary.json
22
  04b007131663760bf3e581e5a953be77044014e87efe1d2a6ca4b72ec0eac978 chat_template.jinja
23
  bb33f3a52d787e15db7d516c6de36a7ddd2c675e7964d5b9309131c4e5d365f0 config.json
24
  f5b7248f7781412e8b5d8f2fb3b0c309568e4e75baf55029b26a6fa985d9d608 distillation/README.md
 
36
  d9b135cf5b0adf4bbfc321200395fbd6112e49fd1a027b30262bcff1c4c0b654 distillation/qwen35_distill/losses.py
37
  00731ae7838317105f9fff960e3da210b2a55954feb37b3ac793bcfc81fdab2d distillation/qwen35_distill/schema.py
38
  4dc432acb70e03848d2e5964e8550065490fda3857aac32fe3b79895ef72026f distillation/qwen35_distill/training.py
39
+ 1b6e475b9d7d9af52bb8845de020303b1c6e89646d8e4f9be17dbe4ab2910417 docs/RELEASE_CONTRACT.md
40
  a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d merges.txt
41
  2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3 model.safetensors
42
  77fd4710def9ec3c0f6225800e0235f15a425abd4a8b03559127fcd782612049 models/semeval-propaganda/LICENSE
43
+ bf4ba7cc2514502a28f424507c41c1cfce5089c051ba7500cf46c1d13860722f models/semeval-propaganda/README.md
44
  0bae347bb54f8089a5fa5450f8366b5c56eb3e598beaf9a87f4385363a402bcb models/semeval-propaganda/benchmark/final4l_aggregate.json
45
  c1bd379680abcf50b47b7d23e4aafbc8b5e7f40a0b0e9700bde21a8ce49af803 models/semeval-propaganda/benchmark/seed41.json
46
  a30e38433d008db4aa7e5fbfd477ab80a0daf50e970f8e7b04397efc1a47843a models/semeval-propaganda/benchmark/seed42.json
 
70
  b9ce99b993c4f4563bc2521fc731bfd82f5e8149c141133599fbd8f982ba43d9 provenance/task_agnostic_stages/24to8.json
71
  ec734855870758cbfef99031dac2ded85b54c496532e5c514a9a4f63287c3493 provenance/task_agnostic_stages/6to4.json
72
  65e5383afbbcfe0b7dd78a5411c319ed6bf2586659fb1892aca9ecbd5dfd4993 provenance/task_agnostic_stages/8to6.json
73
+ 207e1df5765c838b7d355a0c34cf5ae2c55f90b419cff0f4bbf80528a48fd939 release_manifest.json
74
  7b48e7811e56e552fc4aef44189e1d7c9b1352c85d2b3f10ff0bdd1ea5cdd824 requirements.txt
75
  06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523 tokenizer.json
76
  5ab9bed0a4d27949672f65ba1141d6dd5b0514fb9091d3426506ecebb5d5e294 tokenizer_config.json
benchmark/BENCHMARK_CARD.md ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Complete frozen transfer benchmark
2
+
3
+ This report is generated from the nine frozen seed reports distributed in
4
+ `benchmark/reports/`. It evaluates a headless base through fresh-head
5
+ fine-tuning; it is not a zero-shot root-model score.
6
+
7
+ ## Scope
8
+
9
+ - Derived task: article-level presence of 14 propaganda techniques.
10
+ - Test: 55 previously opened English articles / 434 windows.
11
+ - Seeds: 41, 42, 43.
12
+ - Error bars: three-seed sample SD, not confidence intervals.
13
+ - No multilingual downstream evaluation was performed.
14
+
15
+ ## Overall metrics
16
+
17
+ | Arm | Macro-F1 | Micro-F1 | Exact match | Positive rate |
18
+ |---|---:|---:|---:|---:|
19
+ | Task-agnostic KD base | 0.58748 ± 0.01579 | 0.65295 ± 0.01088 | 0.03030 ± 0.02777 | 0.49870 ± 0.08248 |
20
+ | Structural copy, no task-free KD | 0.51809 ± 0.01631 | 0.59046 ± 0.01603 | 0.00000 ± 0.00000 | 0.64329 ± 0.06161 |
21
+ | Existing specialized 4L, separate lineage | 0.58757 ± 0.00717 | 0.64798 ± 0.00187 | 0.00606 ± 0.01050 | 0.58052 ± 0.02848 |
22
+
23
+ The observed mean same-seed Macro-F1 delta between the task-agnostic
24
+ base and structural control is `+0.06939`.
25
+ The specialized arm is a separate historical lineage and is reference-only.
26
+
27
+ ## Task-agnostic base per-label metrics
28
+
29
+ | Label | Support | Precision | Recall | F1 |
30
+ |---|---:|---:|---:|---:|
31
+ | `Appeal_to_Authority` | 11 | 0.3959 | 0.5758 | 0.4657 |
32
+ | `Appeal_to_fear-prejudice` | 24 | 0.5540 | 0.7361 | 0.6278 |
33
+ | `Bandwagon,Reductio_ad_hitlerum` | 7 | 0.4852 | 0.8095 | 0.6007 |
34
+ | `Black-and-White_Fallacy` | 12 | 0.3668 | 0.4722 | 0.4054 |
35
+ | `Causal_Oversimplification` | 18 | 0.5429 | 0.7963 | 0.6447 |
36
+ | `Doubt` | 22 | 0.5224 | 0.9848 | 0.6802 |
37
+ | `Exaggeration,Minimisation` | 22 | 0.4621 | 0.7727 | 0.5727 |
38
+ | `Flag-Waving` | 17 | 0.5643 | 0.8235 | 0.6465 |
39
+ | `Loaded_Language` | 45 | 0.8649 | 0.9852 | 0.9209 |
40
+ | `Name_Calling,Labeling` | 33 | 0.7381 | 0.9091 | 0.8125 |
41
+ | `Repetition` | 23 | 0.5197 | 0.7536 | 0.6148 |
42
+ | `Slogans` | 14 | 0.5161 | 0.7381 | 0.5835 |
43
+ | `Thought-terminating_Cliches` | 7 | 0.3492 | 0.3333 | 0.3190 |
44
+ | `Whataboutism,Straw_Men,Red_Herring` | 10 | 0.2762 | 0.5000 | 0.3304 |
45
+
46
+ ## Resource scope
47
+
48
+ Resource rows describe the fresh-head transfer checkpoints on the recorded
49
+ RTX 5070 Ti BF16 sliding-window protocol. They are not a headless-root CPU
50
+ or edge-device benchmark. See `resource_metrics.csv` for every seed.
51
+
52
+ ## Files
53
+
54
+ - `summary.json`: aggregate and interpretation contract.
55
+ - `seed_metrics.csv`: overall metrics and thresholds by arm/seed.
56
+ - `arm_summary.csv`: means and sample SDs.
57
+ - `per_label_metrics.csv`: precision/recall/F1/support by arm/seed/label.
58
+ - `per_label_summary.csv`: per-label means and sample SDs.
59
+ - `resource_metrics.csv`: parameters, timing, and CUDA memory.
60
+ - `compression_ladder.csv`: 24L→8L→6L→4L size and KD evidence.
61
+ - `reports/`: normalized full frozen reports, including probabilities.
62
+
63
+ ## Interpretation boundary
64
+
65
+ The opened test and prior model-selection history prevent confirmatory
66
+ inference. Three seeds do not represent 165 independent observations.
67
+ Upstream support for 201 languages/dialects motivates multilingual metadata,
68
+ but this four-layer root has no direct multilingual downstream score.
benchmark/arm_summary.csv ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ arm,metric,values,mean,sample_sd,n
2
+ task_agnostic_base,macro_f1,"[0.5883093293517536, 0.6028379036218096, 0.5712816197954101]",0.5874762842563245,0.01579462678670608,3
3
+ task_agnostic_base,micro_f1,"[0.6468085106382978, 0.6655172413793103, 0.6465256797583082]",0.6529504772586388,0.010884055708931178,3
4
+ task_agnostic_base,exact_match,"[0.05454545454545454, 0.03636363636363636, 0.0]",0.0303030303030303,0.027773186030035393,3
5
+ task_agnostic_base,predicted_positive_rate,"[0.5714285714285714, 0.4090909090909091, 0.5155844155844156]",0.4987012987012987,0.08247520160679181,3
6
+ structural_copy_control,macro_f1,"[0.5169424579262757, 0.5349405779540175, 0.5023778364496152]",0.5180869574433028,0.016311512588628545,3
7
+ structural_copy_control,micro_f1,"[0.5773955773955775, 0.6083445491251682, 0.585635359116022]",0.5904584952122559,0.01602830914762526,3
8
+ structural_copy_control,exact_match,"[0.0, 0.0, 0.0]",0.0,0.0,3
9
+ structural_copy_control,predicted_positive_rate,"[0.712987012987013, 0.6207792207792208, 0.5961038961038961]",0.6432900432900432,0.06160737456395461,3
10
+ existing_specialized_separate_lineage,macro_f1,"[0.5832848452288824, 0.5958405181160762, 0.5835785045218137]",0.5875679559555907,0.007165753447433348,3
11
+ existing_specialized_separate_lineage,micro_f1,"[0.6494252873563218, 0.6458616010854816, 0.6486486486486486]",0.647978512363484,0.0018739735696546695,3
12
+ existing_specialized_separate_lineage,exact_match,"[0.0, 0.0, 0.01818181818181818]",0.006060606060606061,0.01049727762162956,3
13
+ existing_specialized_separate_lineage,predicted_positive_rate,"[0.5597402597402598, 0.612987012987013, 0.5688311688311688]",0.5805194805194805,0.02848274311618351,3
benchmark/compression_ladder.csv ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ name,layers,parameters,weight_bytes,weight_mib,weight_sha256,kd_stage,mean_total_loss,mean_interface_loss,mean_final_loss
2
+ 24L upstream text teacher,24,752393024,1504825632,1435.1135559082031,30783da4651259c26b8a204821ea426b9526431e0c14da0d1c6fa284e7414e5e,,,,
3
+ 8L task-free KD,8,420318144,840647880,801.7042922973633,086fbaf9a4838ecde56cd3541c9c3212b22d0c5ee745cf2c1c5e0bb31e0ec22c,24to8,0.17622653172702485,0.21562811763578793,0.16626427527853593
4
+ 6L task-free KD,6,377207424,754423424,719.4742431640625,492749c20f77a42f6bc993f0927a82217b94f872b7eacf25612f7520c9bc28ce,8to6,0.03382104352249371,0.04486069045515251,0.031147060785087888
5
+ 4L task-free KD root,4,334096704,668198976,637.2442016601562,2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3,6to4,0.047020394468859195,0.07048421185527332,0.043119939065377366
benchmark/per_label_metrics.csv ADDED
@@ -0,0 +1,127 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ arm,seed,label,support,precision,recall,f1
2
+ task_agnostic_base,41,Appeal_to_Authority,11,0.3333333333333333,0.5454545454545454,0.41379310344827586
3
+ task_agnostic_base,41,Appeal_to_fear-prejudice,24,0.5757575757575758,0.7916666666666666,0.6666666666666667
4
+ task_agnostic_base,41,"Bandwagon,Reductio_ad_hitlerum",7,0.5,0.8571428571428571,0.631578947368421
5
+ task_agnostic_base,41,Black-and-White_Fallacy,12,0.3333333333333333,0.5833333333333334,0.4242424242424242
6
+ task_agnostic_base,41,Causal_Oversimplification,18,0.5,0.6666666666666666,0.5714285714285715
7
+ task_agnostic_base,41,Doubt,22,0.4782608695652174,1.0,0.6470588235294118
8
+ task_agnostic_base,41,"Exaggeration,Minimisation",22,0.4523809523809524,0.8636363636363636,0.59375
9
+ task_agnostic_base,41,Flag-Waving,17,0.4594594594594595,1.0,0.6296296296296297
10
+ task_agnostic_base,41,Loaded_Language,45,0.9183673469387755,1.0,0.9574468085106383
11
+ task_agnostic_base,41,"Name_Calling,Labeling",33,0.7333333333333333,1.0,0.846153846153846
12
+ task_agnostic_base,41,Repetition,23,0.5428571428571428,0.8260869565217391,0.6551724137931034
13
+ task_agnostic_base,41,Slogans,14,0.41379310344827586,0.8571428571428571,0.5581395348837208
14
+ task_agnostic_base,41,Thought-terminating_Cliches,7,0.21428571428571427,0.42857142857142855,0.2857142857142857
15
+ task_agnostic_base,41,"Whataboutism,Straw_Men,Red_Herring",10,0.22857142857142856,0.8,0.3555555555555555
16
+ task_agnostic_base,42,Appeal_to_Authority,11,0.5384615384615384,0.6363636363636364,0.5833333333333334
17
+ task_agnostic_base,42,Appeal_to_fear-prejudice,24,0.56,0.5833333333333334,0.5714285714285714
18
+ task_agnostic_base,42,"Bandwagon,Reductio_ad_hitlerum",7,0.5555555555555556,0.7142857142857143,0.6250000000000001
19
+ task_agnostic_base,42,Black-and-White_Fallacy,12,0.45454545454545453,0.4166666666666667,0.43478260869565216
20
+ task_agnostic_base,42,Causal_Oversimplification,18,0.5769230769230769,0.8333333333333334,0.6818181818181818
21
+ task_agnostic_base,42,Doubt,22,0.6,0.9545454545454546,0.7368421052631579
22
+ task_agnostic_base,42,"Exaggeration,Minimisation",22,0.48148148148148145,0.5909090909090909,0.5306122448979591
23
+ task_agnostic_base,42,Flag-Waving,17,0.7333333333333333,0.6470588235294118,0.6875
24
+ task_agnostic_base,42,Loaded_Language,45,0.8461538461538461,0.9777777777777777,0.9072164948453608
25
+ task_agnostic_base,42,"Name_Calling,Labeling",33,0.7428571428571429,0.7878787878787878,0.7647058823529412
26
+ task_agnostic_base,42,Repetition,23,0.5,0.7391304347826086,0.5964912280701754
27
+ task_agnostic_base,42,Slogans,14,0.75,0.6428571428571429,0.6923076923076924
28
+ task_agnostic_base,42,Thought-terminating_Cliches,7,0.3333333333333333,0.2857142857142857,0.30769230769230765
29
+ task_agnostic_base,42,"Whataboutism,Straw_Men,Red_Herring",10,0.26666666666666666,0.4,0.32
30
+ task_agnostic_base,43,Appeal_to_Authority,11,0.3157894736842105,0.5454545454545454,0.39999999999999997
31
+ task_agnostic_base,43,Appeal_to_fear-prejudice,24,0.5263157894736842,0.8333333333333334,0.6451612903225806
32
+ task_agnostic_base,43,"Bandwagon,Reductio_ad_hitlerum",7,0.4,0.8571428571428571,0.5454545454545455
33
+ task_agnostic_base,43,Black-and-White_Fallacy,12,0.3125,0.4166666666666667,0.35714285714285715
34
+ task_agnostic_base,43,Causal_Oversimplification,18,0.5517241379310345,0.8888888888888888,0.6808510638297872
35
+ task_agnostic_base,43,Doubt,22,0.4888888888888889,1.0,0.6567164179104478
36
+ task_agnostic_base,43,"Exaggeration,Minimisation",22,0.4523809523809524,0.8636363636363636,0.59375
37
+ task_agnostic_base,43,Flag-Waving,17,0.5,0.8235294117647058,0.6222222222222222
38
+ task_agnostic_base,43,Loaded_Language,45,0.8301886792452831,0.9777777777777777,0.8979591836734695
39
+ task_agnostic_base,43,"Name_Calling,Labeling",33,0.7380952380952381,0.9393939393939394,0.8266666666666667
40
+ task_agnostic_base,43,Repetition,23,0.5161290322580645,0.6956521739130435,0.5925925925925926
41
+ task_agnostic_base,43,Slogans,14,0.38461538461538464,0.7142857142857143,0.5
42
+ task_agnostic_base,43,Thought-terminating_Cliches,7,0.5,0.2857142857142857,0.36363636363636365
43
+ task_agnostic_base,43,"Whataboutism,Straw_Men,Red_Herring",10,0.3333333333333333,0.3,0.3157894736842105
44
+ structural_copy_control,41,Appeal_to_Authority,11,0.2962962962962963,0.7272727272727273,0.42105263157894735
45
+ structural_copy_control,41,Appeal_to_fear-prejudice,24,0.4666666666666667,0.875,0.608695652173913
46
+ structural_copy_control,41,"Bandwagon,Reductio_ad_hitlerum",7,0.2857142857142857,0.8571428571428571,0.42857142857142855
47
+ structural_copy_control,41,Black-and-White_Fallacy,12,0.2222222222222222,0.3333333333333333,0.26666666666666666
48
+ structural_copy_control,41,Causal_Oversimplification,18,0.43243243243243246,0.8888888888888888,0.5818181818181818
49
+ structural_copy_control,41,Doubt,22,0.4,1.0,0.5714285714285715
50
+ structural_copy_control,41,"Exaggeration,Minimisation",22,0.425531914893617,0.9090909090909091,0.5797101449275361
51
+ structural_copy_control,41,Flag-Waving,17,0.37777777777777777,1.0,0.5483870967741935
52
+ structural_copy_control,41,Loaded_Language,45,0.8333333333333334,1.0,0.9090909090909091
53
+ structural_copy_control,41,"Name_Calling,Labeling",33,0.6,1.0,0.7499999999999999
54
+ structural_copy_control,41,Repetition,23,0.46,1.0,0.6301369863013699
55
+ structural_copy_control,41,Slogans,14,0.2857142857142857,0.5714285714285714,0.38095238095238093
56
+ structural_copy_control,41,Thought-terminating_Cliches,7,0.125,0.5714285714285714,0.20512820512820512
57
+ structural_copy_control,41,"Whataboutism,Straw_Men,Red_Herring",10,0.22857142857142856,0.8,0.3555555555555555
58
+ structural_copy_control,42,Appeal_to_Authority,11,0.1875,0.2727272727272727,0.2222222222222222
59
+ structural_copy_control,42,Appeal_to_fear-prejudice,24,0.5384615384615384,0.875,0.6666666666666667
60
+ structural_copy_control,42,"Bandwagon,Reductio_ad_hitlerum",7,0.35,1.0,0.5185185185185185
61
+ structural_copy_control,42,Black-and-White_Fallacy,12,0.38461538461538464,0.8333333333333334,0.5263157894736842
62
+ structural_copy_control,42,Causal_Oversimplification,18,0.4473684210526316,0.9444444444444444,0.6071428571428572
63
+ structural_copy_control,42,Doubt,22,0.4583333333333333,1.0,0.6285714285714286
64
+ structural_copy_control,42,"Exaggeration,Minimisation",22,0.46511627906976744,0.9090909090909091,0.6153846153846153
65
+ structural_copy_control,42,Flag-Waving,17,0.43333333333333335,0.7647058823529411,0.5531914893617021
66
+ structural_copy_control,42,Loaded_Language,45,0.8490566037735849,1.0,0.9183673469387755
67
+ structural_copy_control,42,"Name_Calling,Labeling",33,0.6666666666666666,0.9696969696969697,0.7901234567901234
68
+ structural_copy_control,42,Repetition,23,0.43137254901960786,0.9565217391304348,0.5945945945945946
69
+ structural_copy_control,42,Slogans,14,0.3888888888888889,0.5,0.43750000000000006
70
+ structural_copy_control,42,Thought-terminating_Cliches,7,0.11764705882352941,0.2857142857142857,0.16666666666666666
71
+ structural_copy_control,42,"Whataboutism,Straw_Men,Red_Herring",10,0.16129032258064516,0.5,0.24390243902439024
72
+ structural_copy_control,43,Appeal_to_Authority,11,0.2962962962962963,0.7272727272727273,0.42105263157894735
73
+ structural_copy_control,43,Appeal_to_fear-prejudice,24,0.5263157894736842,0.8333333333333334,0.6451612903225806
74
+ structural_copy_control,43,"Bandwagon,Reductio_ad_hitlerum",7,0.22727272727272727,0.7142857142857143,0.3448275862068965
75
+ structural_copy_control,43,Black-and-White_Fallacy,12,0.3333333333333333,0.4166666666666667,0.3703703703703704
76
+ structural_copy_control,43,Causal_Oversimplification,18,0.4594594594594595,0.9444444444444444,0.6181818181818182
77
+ structural_copy_control,43,Doubt,22,0.4782608695652174,1.0,0.6470588235294118
78
+ structural_copy_control,43,"Exaggeration,Minimisation",22,0.4186046511627907,0.8181818181818182,0.5538461538461539
79
+ structural_copy_control,43,Flag-Waving,17,0.3333333333333333,0.6470588235294118,0.44
80
+ structural_copy_control,43,Loaded_Language,45,0.8653846153846154,1.0,0.9278350515463918
81
+ structural_copy_control,43,"Name_Calling,Labeling",33,0.6458333333333334,0.9393939393939394,0.7654320987654321
82
+ structural_copy_control,43,Repetition,23,0.45454545454545453,0.8695652173913043,0.5970149253731344
83
+ structural_copy_control,43,Slogans,14,0.3076923076923077,0.2857142857142857,0.29629629629629634
84
+ structural_copy_control,43,Thought-terminating_Cliches,7,0.1,0.2857142857142857,0.14814814814814817
85
+ structural_copy_control,43,"Whataboutism,Straw_Men,Red_Herring",10,0.19047619047619047,0.4,0.25806451612903225
86
+ existing_specialized_separate_lineage,41,Appeal_to_Authority,11,0.5,0.6363636363636364,0.56
87
+ existing_specialized_separate_lineage,41,Appeal_to_fear-prejudice,24,0.5,0.9166666666666666,0.6470588235294118
88
+ existing_specialized_separate_lineage,41,"Bandwagon,Reductio_ad_hitlerum",7,0.4444444444444444,0.5714285714285714,0.5
89
+ existing_specialized_separate_lineage,41,Black-and-White_Fallacy,12,0.4666666666666667,0.5833333333333334,0.5185185185185186
90
+ existing_specialized_separate_lineage,41,Causal_Oversimplification,18,0.46153846153846156,0.6666666666666666,0.5454545454545455
91
+ existing_specialized_separate_lineage,41,Doubt,22,0.5238095238095238,1.0,0.6875000000000001
92
+ existing_specialized_separate_lineage,41,"Exaggeration,Minimisation",22,0.4418604651162791,0.8636363636363636,0.5846153846153846
93
+ existing_specialized_separate_lineage,41,Flag-Waving,17,0.53125,1.0,0.6938775510204082
94
+ existing_specialized_separate_lineage,41,Loaded_Language,45,0.8490566037735849,1.0,0.9183673469387755
95
+ existing_specialized_separate_lineage,41,"Name_Calling,Labeling",33,0.717391304347826,1.0,0.8354430379746834
96
+ existing_specialized_separate_lineage,41,Repetition,23,0.46938775510204084,1.0,0.6388888888888888
97
+ existing_specialized_separate_lineage,41,Slogans,14,0.5,0.5,0.5
98
+ existing_specialized_separate_lineage,41,Thought-terminating_Cliches,7,0.14285714285714285,0.5714285714285714,0.2285714285714286
99
+ existing_specialized_separate_lineage,41,"Whataboutism,Straw_Men,Red_Herring",10,0.25,0.4,0.3076923076923077
100
+ existing_specialized_separate_lineage,42,Appeal_to_Authority,11,0.3225806451612903,0.9090909090909091,0.4761904761904761
101
+ existing_specialized_separate_lineage,42,Appeal_to_fear-prejudice,24,0.5405405405405406,0.8333333333333334,0.6557377049180328
102
+ existing_specialized_separate_lineage,42,"Bandwagon,Reductio_ad_hitlerum",7,0.42857142857142855,0.8571428571428571,0.5714285714285714
103
+ existing_specialized_separate_lineage,42,Black-and-White_Fallacy,12,0.47058823529411764,0.6666666666666666,0.5517241379310345
104
+ existing_specialized_separate_lineage,42,Causal_Oversimplification,18,0.4444444444444444,0.8888888888888888,0.5925925925925926
105
+ existing_specialized_separate_lineage,42,Doubt,22,0.5116279069767442,1.0,0.676923076923077
106
+ existing_specialized_separate_lineage,42,"Exaggeration,Minimisation",22,0.4375,0.9545454545454546,0.6
107
+ existing_specialized_separate_lineage,42,Flag-Waving,17,0.6,0.8823529411764706,0.7142857142857143
108
+ existing_specialized_separate_lineage,42,Loaded_Language,45,0.8181818181818182,1.0,0.9
109
+ existing_specialized_separate_lineage,42,"Name_Calling,Labeling",33,0.6808510638297872,0.9696969696969697,0.7999999999999999
110
+ existing_specialized_separate_lineage,42,Repetition,23,0.4888888888888889,0.9565217391304348,0.6470588235294117
111
+ existing_specialized_separate_lineage,42,Slogans,14,0.39285714285714285,0.7857142857142857,0.5238095238095237
112
+ existing_specialized_separate_lineage,42,Thought-terminating_Cliches,7,0.2,0.8571428571428571,0.32432432432432434
113
+ existing_specialized_separate_lineage,42,"Whataboutism,Straw_Men,Red_Herring",10,0.25,0.4,0.3076923076923077
114
+ existing_specialized_separate_lineage,43,Appeal_to_Authority,11,0.3125,0.45454545454545453,0.3703703703703703
115
+ existing_specialized_separate_lineage,43,Appeal_to_fear-prejudice,24,0.5365853658536586,0.9166666666666666,0.676923076923077
116
+ existing_specialized_separate_lineage,43,"Bandwagon,Reductio_ad_hitlerum",7,0.46153846153846156,0.8571428571428571,0.6
117
+ existing_specialized_separate_lineage,43,Black-and-White_Fallacy,12,0.46153846153846156,0.5,0.48000000000000004
118
+ existing_specialized_separate_lineage,43,Causal_Oversimplification,18,0.4358974358974359,0.9444444444444444,0.5964912280701755
119
+ existing_specialized_separate_lineage,43,Doubt,22,0.4782608695652174,1.0,0.6470588235294118
120
+ existing_specialized_separate_lineage,43,"Exaggeration,Minimisation",22,0.4583333333333333,1.0,0.6285714285714286
121
+ existing_specialized_separate_lineage,43,Flag-Waving,17,0.5416666666666666,0.7647058823529411,0.6341463414634146
122
+ existing_specialized_separate_lineage,43,Loaded_Language,45,0.8490566037735849,1.0,0.9183673469387755
123
+ existing_specialized_separate_lineage,43,"Name_Calling,Labeling",33,0.6888888888888889,0.9393939393939394,0.7948717948717948
124
+ existing_specialized_separate_lineage,43,Repetition,23,0.47619047619047616,0.8695652173913043,0.6153846153846153
125
+ existing_specialized_separate_lineage,43,Slogans,14,0.4074074074074074,0.7857142857142857,0.5365853658536585
126
+ existing_specialized_separate_lineage,43,Thought-terminating_Cliches,7,0.21052631578947367,0.5714285714285714,0.3076923076923077
127
+ existing_specialized_separate_lineage,43,"Whataboutism,Straw_Men,Red_Herring",10,0.3333333333333333,0.4,0.3636363636363636
benchmark/per_label_summary.csv ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ arm,label,support,precision_mean,precision_sample_sd,recall_mean,recall_sample_sd,f1_mean,f1_sample_sd
2
+ task_agnostic_base,Appeal_to_Authority,11,0.3958614484930274,0.1238064457010119,0.5757575757575757,0.05248638810814781,0.4657088122605364,0.1020990126979069
3
+ task_agnostic_base,Appeal_to_fear-prejudice,24,0.5540244550770866,0.02525673990402998,0.7361111111111112,0.13393959390267993,0.6277521761392729,0.04994879016183189
4
+ task_agnostic_base,"Bandwagon,Reductio_ad_hitlerum",7,0.4851851851851852,0.07882887649552875,0.8095238095238095,0.08247860988423221,0.6006778309409889,0.04793776258918466
5
+ task_agnostic_base,Black-and-White_Fallacy,12,0.3667929292929293,0.07670649392711647,0.47222222222222227,0.09622504486493764,0.4053892966936445,0.04211369218048746
6
+ task_agnostic_base,Causal_Oversimplification,18,0.5428824049513704,0.039216350192122156,0.7962962962962963,0.11564811108145183,0.6446992723588468,0.0634561308339175
7
+ task_agnostic_base,Doubt,22,0.5223832528180354,0.06742780050556177,0.9848484848484849,0.026243194054073875,0.6802057822343391,0.04928561754686331
8
+ task_agnostic_base,"Exaggeration,Minimisation",22,0.4620811287477954,0.016801198309751,0.7727272727272727,0.15745916432444337,0.5727040816326531,0.03645259990419198
9
+ task_agnostic_base,Flag-Waving,17,0.5642642642642642,0.1478145677656334,0.8235294117647058,0.1764705882352941,0.6464506172839506,0.035742219952758225
10
+ task_agnostic_base,Loaded_Language,45,0.8649032907793016,0.046984312428427724,0.9851851851851852,0.012830005981991702,0.9208741623431562,0.03200926878737141
11
+ task_agnostic_base,"Name_Calling,Labeling",33,0.7380952380952381,0.0047619047619048005,0.9090909090909091,0.10925912955951485,0.8125087983911513,0.04252971464057227
12
+ task_agnostic_base,Repetition,23,0.5196620583717357,0.021645909050194662,0.7536231884057971,0.06641414050660639,0.6147520781519571,0.03505927109416221
13
+ task_agnostic_base,Slogans,14,0.5161361626878869,0.20305677930281646,0.7380952380952381,0.10910894511799614,0.5834824090638044,0.09862686289220693
14
+ task_agnostic_base,Thought-terminating_Cliches,7,0.3492063492063492,0.143516994603374,0.3333333333333333,0.08247860988423225,0.319014319014319,0.04017590711117064
15
+ task_agnostic_base,"Whataboutism,Straw_Men,Red_Herring",10,0.2761904761904762,0.05302632726504782,0.5,0.2645751311064591,0.330448343079922,0.021845164720973014
16
+ structural_copy_control,Appeal_to_Authority,11,0.26003086419753085,0.06281357095350094,0.5757575757575758,0.262431940540739,0.35477582846003897,0.11479479036518875
17
+ structural_copy_control,Appeal_to_fear-prejudice,24,0.5104813315339631,0.038427508996734555,0.8611111111111112,0.024056261216234387,0.6401745363877202,0.029305467347150463
18
+ structural_copy_control,"Bandwagon,Reductio_ad_hitlerum",7,0.2876623376623376,0.061386823077618426,0.8571428571428571,0.14285714285714285,0.4306391777656145,0.08686392624022733
19
+ structural_copy_control,Black-and-White_Fallacy,12,0.31339031339031337,0.08301311842925053,0.5277777777777778,0.2678791878053599,0.38778427550357375,0.13069755090515112
20
+ structural_copy_control,Causal_Oversimplification,18,0.4464201043148412,0.013538446216471536,0.9259259259259259,0.03207501495497923,0.6023809523809524,0.018643640071557464
21
+ structural_copy_control,Doubt,22,0.4455314009661836,0.04067072694857813,1.0,0.0,0.615686274509804,0.039427200694482675
22
+ structural_copy_control,"Exaggeration,Minimisation",22,0.4364176150420584,0.025093958106097894,0.8787878787878788,0.05248638810814775,0.5829803047194351,0.030899288356345502
23
+ structural_copy_control,Flag-Waving,17,0.3814814814814815,0.05010277503136551,0.803921568627451,0.17970885078258195,0.5138595287119652,0.06400931989326974
24
+ structural_copy_control,Loaded_Language,45,0.8492581841638446,0.016026591848331686,1.0,0.0,0.9184311025253588,0.009372233868132662
25
+ structural_copy_control,"Name_Calling,Labeling",33,0.6375,0.03410563654946854,0.9696969696969697,0.030303030303030276,0.7685185185185185,0.02023900779105559
26
+ structural_copy_control,Repetition,23,0.44863933452168747,0.01520014564765896,0.9420289855072463,0.06641414050660639,0.607248835423033,0.019858627551810427
27
+ structural_copy_control,Slogans,14,0.3274318274318274,0.05434598572773684,0.4523809523809524,0.1486904285332952,0.3715828924162258,0.07106660236343577
28
+ structural_copy_control,Thought-terminating_Cliches,7,0.1142156862745098,0.012848374923608973,0.38095238095238093,0.1649572197684645,0.17331433998100665,0.029065880233243847
29
+ structural_copy_control,"Whataboutism,Straw_Men,Red_Herring",10,0.19344598054275472,0.03373872471827226,0.5666666666666667,0.2081665999466133,0.28584083690299267,0.06078854830133676
30
+ existing_specialized_separate_lineage,Appeal_to_Authority,11,0.37836021505376344,0.10546365636105155,0.6666666666666666,0.2287828616748712,0.46885361552028215,0.09502747629890008
31
+ existing_specialized_separate_lineage,Appeal_to_fear-prejudice,24,0.525708635464733,0.02235198659615819,0.8888888888888888,0.048112522432468774,0.6599065351235072,0.015362381542320084
32
+ existing_specialized_separate_lineage,"Bandwagon,Reductio_ad_hitlerum",7,0.44485144485144484,0.016487284575115283,0.7619047619047619,0.1649572197684645,0.5571428571428572,0.051507875363771265
33
+ existing_specialized_separate_lineage,Black-and-White_Fallacy,12,0.4662644544997486,0.004538274146363688,0.5833333333333334,0.08333333333333331,0.516747552149851,0.03589484965697345
34
+ existing_specialized_separate_lineage,Causal_Oversimplification,18,0.4472934472934473,0.013055771210700413,0.8333333333333333,0.1469861839480328,0.5781794553724379,0.028407563015813122
35
+ existing_specialized_separate_lineage,Doubt,22,0.5045661001171619,0.023581174969843577,1.0,0.0,0.6704939668174963,0.020973132363659372
36
+ existing_specialized_separate_lineage,"Exaggeration,Minimisation",22,0.44589793281653745,0.010987847993006481,0.9393939393939394,0.06943296507508849,0.6043956043956044,0.022305256187015846
37
+ existing_specialized_separate_lineage,Flag-Waving,17,0.5576388888888889,0.03705367103320583,0.8823529411764706,0.11764705882352944,0.680769868923179,0.04164658769406446
38
+ existing_specialized_separate_lineage,Loaded_Language,45,0.8387650085763294,0.01782556577257849,1.0,0.0,0.9122448979591837,0.01060439269940129
39
+ existing_specialized_separate_lineage,"Name_Calling,Labeling",33,0.6957104190221673,0.019201490809269257,0.9696969696969697,0.030303030303030276,0.8101049442821594,0.022092733328802537
40
+ existing_specialized_separate_lineage,Repetition,23,0.4781557067271353,0.009897987322673213,0.9420289855072463,0.06641414050660639,0.6337774426009719,0.016444118333485254
41
+ existing_specialized_separate_lineage,Slogans,14,0.4334215167548501,0.05811581861282867,0.6904761904761905,0.1649572197684645,0.5201316298877274,0.018567913895656114
42
+ existing_specialized_separate_lineage,Thought-terminating_Cliches,7,0.18446115288220552,0.03641251252864963,0.6666666666666666,0.1649572197684645,0.2868626868626869,0.0511620864429468
43
+ existing_specialized_separate_lineage,"Whataboutism,Straw_Men,Red_Herring",10,0.2777777777777778,0.0481125224324688,0.4,0.0,0.32634032634032634,0.03229931575886014
benchmark/reports/existing_specialized_separate_lineage-seed41.json ADDED
The diff for this file is too large to render. See raw diff
 
benchmark/reports/existing_specialized_separate_lineage-seed42.json ADDED
The diff for this file is too large to render. See raw diff
 
benchmark/reports/existing_specialized_separate_lineage-seed43.json ADDED
The diff for this file is too large to render. See raw diff
 
benchmark/reports/structural_copy_control-seed41.json ADDED
The diff for this file is too large to render. See raw diff
 
benchmark/reports/structural_copy_control-seed42.json ADDED
The diff for this file is too large to render. See raw diff
 
benchmark/reports/structural_copy_control-seed43.json ADDED
The diff for this file is too large to render. See raw diff
 
benchmark/reports/task_agnostic_base-seed41.json ADDED
The diff for this file is too large to render. See raw diff
 
benchmark/reports/task_agnostic_base-seed42.json ADDED
The diff for this file is too large to render. See raw diff
 
benchmark/reports/task_agnostic_base-seed43.json ADDED
The diff for this file is too large to render. See raw diff
 
benchmark/resource_metrics.csv ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ arm,seed,parameters,trainable_parameters,peak_cuda_allocated_bytes,peak_cuda_reserved_bytes,total_wall_seconds,test_wall_seconds,test_seconds_per_article,device,dtype
2
+ task_agnostic_base,41,334111040,334111040,3380975616,3519021056,1583.0297977919981,4.490653285000008,0.08164824154545468,NVIDIA GeForce RTX 5070 Ti,bfloat16
3
+ task_agnostic_base,42,334111040,334111040,3380975616,3519021056,1588.617994428001,4.329529417009326,0.07871871667289683,NVIDIA GeForce RTX 5070 Ti,bfloat16
4
+ task_agnostic_base,43,334111040,334111040,3380975616,3510632448,1603.4310500120046,4.595363147003809,0.08355205721825107,NVIDIA GeForce RTX 5070 Ti,bfloat16
5
+ structural_copy_control,41,334111040,334111040,3380975616,3519021056,1575.5905925639963,4.332684432010865,0.07877608058201573,NVIDIA GeForce RTX 5070 Ti,bfloat16
6
+ structural_copy_control,42,334111040,334111040,3380975616,3519021056,1623.5639530190092,4.569082793022972,0.08307423260041767,NVIDIA GeForce RTX 5070 Ti,bfloat16
7
+ structural_copy_control,43,334111040,334111040,3380975616,3510632448,1629.195114912989,4.4899811029899865,0.08163602005436339,NVIDIA GeForce RTX 5070 Ti,bfloat16
8
+ existing_specialized_separate_lineage,41,210903360,210903360,2143655936,2281701376,1634.488557530014,4.6637764379847795,0.0847959352360869,NVIDIA GeForce RTX 5070 Ti,bfloat16
9
+ existing_specialized_separate_lineage,42,210903360,210903360,2143655936,2281701376,1652.9397718419787,4.728947311989032,0.08598086021798239,NVIDIA GeForce RTX 5070 Ti,bfloat16
10
+ existing_specialized_separate_lineage,43,210903360,210903360,2143655936,2273312768,1643.488202479988,4.704143202980049,0.08552987641781908,NVIDIA GeForce RTX 5070 Ti,bfloat16
benchmark/seed_metrics.csv ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ arm,seed,threshold,macro_f1,micro_f1,exact_match,predicted_positive_rate,tp,fp,fn,test_articles,test_windows
2
+ task_agnostic_base,41,0.15,0.5883093293517536,0.6468085106382978,0.05454545454545454,0.5714285714285714,228,212,37,55,434
3
+ task_agnostic_base,42,0.3,0.6028379036218096,0.6655172413793103,0.03636363636363636,0.4090909090909091,193,122,72,55,434
4
+ task_agnostic_base,43,0.1,0.5712816197954101,0.6465256797583082,0.0,0.5155844155844156,214,183,51,55,434
5
+ structural_copy_control,41,0.05,0.5169424579262757,0.5773955773955775,0.0,0.712987012987013,235,314,30,55,434
6
+ structural_copy_control,42,0.2,0.5349405779540175,0.6083445491251682,0.0,0.6207792207792208,226,252,39,55,434
7
+ structural_copy_control,43,0.2,0.5023778364496152,0.585635359116022,0.0,0.5961038961038961,212,247,53,55,434
8
+ existing_specialized_separate_lineage,41,0.1,0.5832848452288824,0.6494252873563218,0.0,0.5597402597402598,226,205,39,55,434
9
+ existing_specialized_separate_lineage,42,0.05,0.5958405181160762,0.6458616010854816,0.0,0.612987012987013,238,234,27,55,434
10
+ existing_specialized_separate_lineage,43,0.05,0.5835785045218137,0.6486486486486486,0.01818181818181818,0.5688311688311688,228,210,37,55,434
benchmark/semeval-transfer-summary.json CHANGED
@@ -1,47 +1,891 @@
1
  {
2
- "schema": "standalone4l-transfer-summary-v1",
3
- "claim_scope": "Exploratory one-task transfer evidence on a previously opened 55-article SemEval-derived test split; not confirmatory.",
4
- "metric": "article-level multilabel Macro-F1",
 
 
 
 
5
  "seeds": [
6
  41,
7
  42,
8
  43
9
  ],
10
- "task_agnostic_base": {
11
- "values": [
12
- 0.5883093293517536,
13
- 0.6028379036218096,
14
- 0.5712816197954101
15
- ],
16
- "mean": 0.5874762842563245,
17
- "sample_sd": 0.01579462678670608
18
  },
19
- "structural_copy_control": {
20
- "values": [
21
- 0.5169424579262757,
22
- 0.5349405779540175,
23
- 0.5023778364496152
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
  ],
25
- "mean": 0.5180869574433028,
26
- "sample_sd": 0.016311512588628545
27
  },
28
- "same_seed_deltas": [
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
  0.07136687142547793,
30
  0.06789732566779205,
31
  0.06890378334579494
32
  ],
33
- "mean_delta": 0.06938932681302164,
34
  "publication_gate_passed": true,
35
- "existing_specialized_separate_lineage": {
36
- "values": [
37
- 0.5832848452288824,
38
- 0.5958405181160762,
39
- 0.5835785045218137
40
- ],
41
- "mean": 0.5875679559555907,
42
- "sample_sd": 0.007165593191426643
43
- },
44
- "test_articles": 55,
45
- "test_previously_opened": true,
46
- "three_seed_sd_is_not_confidence_interval": true
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
  }
 
1
  {
2
+ "schema": "standalone4l-complete-transfer-benchmark-v2",
3
+ "status": "complete_from_frozen_reports",
4
+ "claim_scope": "Exploratory fresh-head transfer evidence on one English task and a previously opened 55-article SemEval-derived test split.",
5
+ "headless_root_requires_adaptation": true,
6
+ "test_previously_opened": true,
7
+ "test_articles": 55,
8
+ "test_windows": 434,
9
  "seeds": [
10
  41,
11
  42,
12
  43
13
  ],
14
+ "three_seed_sd_is_not_confidence_interval": true,
15
+ "multilingual_downstream_evaluation_performed": false,
16
+ "upstream_multilingual_metadata": {
17
+ "source": "Qwen/Qwen3.5-0.8B official model card",
18
+ "languages_and_dialects_claimed_upstream": 201,
19
+ "scope": "inherited tokenizer and architecture coverage only"
 
 
20
  },
21
+ "protocol": {
22
+ "epochs": 5,
23
+ "primary_epoch": 5,
24
+ "early_stopping": false,
25
+ "max_length": 256,
26
+ "stride": 128,
27
+ "window_target": "span-aligned technique labels",
28
+ "article_aggregation": "labelwise maximum probability",
29
+ "micro_batch_size": 1,
30
+ "gradient_accumulation": 32,
31
+ "effective_batch_size": 32,
32
+ "learning_rate": 2e-05,
33
+ "weight_decay": 0.01,
34
+ "gradient_clip": 1.0,
35
+ "optimizer": "AdamW",
36
+ "dtype": "bfloat16",
37
+ "thresholds": [
38
+ 0.05,
39
+ 0.1,
40
+ 0.15,
41
+ 0.2,
42
+ 0.25,
43
+ 0.3,
44
+ 0.35,
45
+ 0.4,
46
+ 0.45,
47
+ 0.5,
48
+ 0.55,
49
+ 0.6,
50
+ 0.65,
51
+ 0.7,
52
+ 0.75,
53
+ 0.8,
54
+ 0.85,
55
+ 0.9
56
  ],
57
+ "threshold_selection": "epoch-5 calibration macro-F1, then micro-F1"
 
58
  },
59
+ "arms": {
60
+ "task_agnostic_base": {
61
+ "display": "Task-agnostic KD base",
62
+ "lineage": "root base plus a fresh 14-label transfer head",
63
+ "metrics": {
64
+ "macro_f1": {
65
+ "values": [
66
+ 0.5883093293517536,
67
+ 0.6028379036218096,
68
+ 0.5712816197954101
69
+ ],
70
+ "mean": 0.5874762842563245,
71
+ "sample_sd": 0.01579462678670608,
72
+ "n": 3
73
+ },
74
+ "micro_f1": {
75
+ "values": [
76
+ 0.6468085106382978,
77
+ 0.6655172413793103,
78
+ 0.6465256797583082
79
+ ],
80
+ "mean": 0.6529504772586388,
81
+ "sample_sd": 0.010884055708931178,
82
+ "n": 3
83
+ },
84
+ "exact_match": {
85
+ "values": [
86
+ 0.05454545454545454,
87
+ 0.03636363636363636,
88
+ 0.0
89
+ ],
90
+ "mean": 0.0303030303030303,
91
+ "sample_sd": 0.027773186030035393,
92
+ "n": 3
93
+ },
94
+ "predicted_positive_rate": {
95
+ "values": [
96
+ 0.5714285714285714,
97
+ 0.4090909090909091,
98
+ 0.5155844155844156
99
+ ],
100
+ "mean": 0.4987012987012987,
101
+ "sample_sd": 0.08247520160679181,
102
+ "n": 3
103
+ }
104
+ },
105
+ "resources": {
106
+ "parameters": {
107
+ "values": [
108
+ 334111040.0,
109
+ 334111040.0,
110
+ 334111040.0
111
+ ],
112
+ "mean": 334111040.0,
113
+ "sample_sd": 0.0,
114
+ "n": 3
115
+ },
116
+ "peak_cuda_allocated_bytes": {
117
+ "values": [
118
+ 3380975616.0,
119
+ 3380975616.0,
120
+ 3380975616.0
121
+ ],
122
+ "mean": 3380975616.0,
123
+ "sample_sd": 0.0,
124
+ "n": 3
125
+ },
126
+ "peak_cuda_reserved_bytes": {
127
+ "values": [
128
+ 3519021056.0,
129
+ 3519021056.0,
130
+ 3510632448.0
131
+ ],
132
+ "mean": 3516224853.3333335,
133
+ "sample_sd": 4843165.086926248,
134
+ "n": 3
135
+ },
136
+ "total_wall_seconds": {
137
+ "values": [
138
+ 1583.0297977919981,
139
+ 1588.617994428001,
140
+ 1603.4310500120046
141
+ ],
142
+ "mean": 1591.692947410668,
143
+ "sample_sd": 10.54249851412311,
144
+ "n": 3
145
+ },
146
+ "test_seconds_per_article": {
147
+ "values": [
148
+ 0.08164824154545468,
149
+ 0.07871871667289683,
150
+ 0.08355205721825107
151
+ ],
152
+ "mean": 0.08130633847886752,
153
+ "sample_sd": 0.0024347419754560395,
154
+ "n": 3
155
+ }
156
+ },
157
+ "thresholds": [
158
+ 0.15,
159
+ 0.3,
160
+ 0.1
161
+ ]
162
+ },
163
+ "structural_copy_control": {
164
+ "display": "Structural copy, no task-free KD",
165
+ "lineage": "same four-layer structure without task-free KD",
166
+ "metrics": {
167
+ "macro_f1": {
168
+ "values": [
169
+ 0.5169424579262757,
170
+ 0.5349405779540175,
171
+ 0.5023778364496152
172
+ ],
173
+ "mean": 0.5180869574433028,
174
+ "sample_sd": 0.016311512588628545,
175
+ "n": 3
176
+ },
177
+ "micro_f1": {
178
+ "values": [
179
+ 0.5773955773955775,
180
+ 0.6083445491251682,
181
+ 0.585635359116022
182
+ ],
183
+ "mean": 0.5904584952122559,
184
+ "sample_sd": 0.01602830914762526,
185
+ "n": 3
186
+ },
187
+ "exact_match": {
188
+ "values": [
189
+ 0.0,
190
+ 0.0,
191
+ 0.0
192
+ ],
193
+ "mean": 0.0,
194
+ "sample_sd": 0.0,
195
+ "n": 3
196
+ },
197
+ "predicted_positive_rate": {
198
+ "values": [
199
+ 0.712987012987013,
200
+ 0.6207792207792208,
201
+ 0.5961038961038961
202
+ ],
203
+ "mean": 0.6432900432900432,
204
+ "sample_sd": 0.06160737456395461,
205
+ "n": 3
206
+ }
207
+ },
208
+ "resources": {
209
+ "parameters": {
210
+ "values": [
211
+ 334111040.0,
212
+ 334111040.0,
213
+ 334111040.0
214
+ ],
215
+ "mean": 334111040.0,
216
+ "sample_sd": 0.0,
217
+ "n": 3
218
+ },
219
+ "peak_cuda_allocated_bytes": {
220
+ "values": [
221
+ 3380975616.0,
222
+ 3380975616.0,
223
+ 3380975616.0
224
+ ],
225
+ "mean": 3380975616.0,
226
+ "sample_sd": 0.0,
227
+ "n": 3
228
+ },
229
+ "peak_cuda_reserved_bytes": {
230
+ "values": [
231
+ 3519021056.0,
232
+ 3519021056.0,
233
+ 3510632448.0
234
+ ],
235
+ "mean": 3516224853.3333335,
236
+ "sample_sd": 4843165.086926248,
237
+ "n": 3
238
+ },
239
+ "total_wall_seconds": {
240
+ "values": [
241
+ 1575.5905925639963,
242
+ 1623.5639530190092,
243
+ 1629.195114912989
244
+ ],
245
+ "mean": 1609.4498868319981,
246
+ "sample_sd": 29.457874198835384,
247
+ "n": 3
248
+ },
249
+ "test_seconds_per_article": {
250
+ "values": [
251
+ 0.07877608058201573,
252
+ 0.08307423260041767,
253
+ 0.08163602005436339
254
+ ],
255
+ "mean": 0.08116211107893226,
256
+ "sample_sd": 0.002187914527825257,
257
+ "n": 3
258
+ }
259
+ },
260
+ "thresholds": [
261
+ 0.05,
262
+ 0.2,
263
+ 0.2
264
+ ]
265
+ },
266
+ "existing_specialized_separate_lineage": {
267
+ "display": "Existing specialized 4L, separate lineage",
268
+ "lineage": "historical task-specific shrink/distillation lineage",
269
+ "metrics": {
270
+ "macro_f1": {
271
+ "values": [
272
+ 0.5832848452288824,
273
+ 0.5958405181160762,
274
+ 0.5835785045218137
275
+ ],
276
+ "mean": 0.5875679559555907,
277
+ "sample_sd": 0.007165753447433348,
278
+ "n": 3
279
+ },
280
+ "micro_f1": {
281
+ "values": [
282
+ 0.6494252873563218,
283
+ 0.6458616010854816,
284
+ 0.6486486486486486
285
+ ],
286
+ "mean": 0.647978512363484,
287
+ "sample_sd": 0.0018739735696546695,
288
+ "n": 3
289
+ },
290
+ "exact_match": {
291
+ "values": [
292
+ 0.0,
293
+ 0.0,
294
+ 0.01818181818181818
295
+ ],
296
+ "mean": 0.006060606060606061,
297
+ "sample_sd": 0.01049727762162956,
298
+ "n": 3
299
+ },
300
+ "predicted_positive_rate": {
301
+ "values": [
302
+ 0.5597402597402598,
303
+ 0.612987012987013,
304
+ 0.5688311688311688
305
+ ],
306
+ "mean": 0.5805194805194805,
307
+ "sample_sd": 0.02848274311618351,
308
+ "n": 3
309
+ }
310
+ },
311
+ "resources": {
312
+ "parameters": {
313
+ "values": [
314
+ 210903360.0,
315
+ 210903360.0,
316
+ 210903360.0
317
+ ],
318
+ "mean": 210903360.0,
319
+ "sample_sd": 0.0,
320
+ "n": 3
321
+ },
322
+ "peak_cuda_allocated_bytes": {
323
+ "values": [
324
+ 2143655936.0,
325
+ 2143655936.0,
326
+ 2143655936.0
327
+ ],
328
+ "mean": 2143655936.0,
329
+ "sample_sd": 0.0,
330
+ "n": 3
331
+ },
332
+ "peak_cuda_reserved_bytes": {
333
+ "values": [
334
+ 2281701376.0,
335
+ 2281701376.0,
336
+ 2273312768.0
337
+ ],
338
+ "mean": 2278905173.3333335,
339
+ "sample_sd": 4843165.086926248,
340
+ "n": 3
341
+ },
342
+ "total_wall_seconds": {
343
+ "values": [
344
+ 1634.488557530014,
345
+ 1652.9397718419787,
346
+ 1643.488202479988
347
+ ],
348
+ "mean": 1643.6388439506602,
349
+ "sample_sd": 9.226529522850198,
350
+ "n": 3
351
+ },
352
+ "test_seconds_per_article": {
353
+ "values": [
354
+ 0.0847959352360869,
355
+ 0.08598086021798239,
356
+ 0.08552987641781908
357
+ ],
358
+ "mean": 0.08543555729062946,
359
+ "sample_sd": 0.0005980667826383295,
360
+ "n": 3
361
+ }
362
+ },
363
+ "thresholds": [
364
+ 0.1,
365
+ 0.05,
366
+ 0.05
367
+ ]
368
+ }
369
+ },
370
+ "same_seed_macro_f1_deltas_base_minus_control": [
371
  0.07136687142547793,
372
  0.06789732566779205,
373
  0.06890378334579494
374
  ],
375
+ "mean_macro_f1_delta_base_minus_control": 0.06938932681302164,
376
  "publication_gate_passed": true,
377
+ "per_label_summary": [
378
+ {
379
+ "arm": "task_agnostic_base",
380
+ "label": "Appeal_to_Authority",
381
+ "support": 11,
382
+ "precision_mean": 0.3958614484930274,
383
+ "precision_sample_sd": 0.1238064457010119,
384
+ "recall_mean": 0.5757575757575757,
385
+ "recall_sample_sd": 0.05248638810814781,
386
+ "f1_mean": 0.4657088122605364,
387
+ "f1_sample_sd": 0.1020990126979069
388
+ },
389
+ {
390
+ "arm": "task_agnostic_base",
391
+ "label": "Appeal_to_fear-prejudice",
392
+ "support": 24,
393
+ "precision_mean": 0.5540244550770866,
394
+ "precision_sample_sd": 0.02525673990402998,
395
+ "recall_mean": 0.7361111111111112,
396
+ "recall_sample_sd": 0.13393959390267993,
397
+ "f1_mean": 0.6277521761392729,
398
+ "f1_sample_sd": 0.04994879016183189
399
+ },
400
+ {
401
+ "arm": "task_agnostic_base",
402
+ "label": "Bandwagon,Reductio_ad_hitlerum",
403
+ "support": 7,
404
+ "precision_mean": 0.4851851851851852,
405
+ "precision_sample_sd": 0.07882887649552875,
406
+ "recall_mean": 0.8095238095238095,
407
+ "recall_sample_sd": 0.08247860988423221,
408
+ "f1_mean": 0.6006778309409889,
409
+ "f1_sample_sd": 0.04793776258918466
410
+ },
411
+ {
412
+ "arm": "task_agnostic_base",
413
+ "label": "Black-and-White_Fallacy",
414
+ "support": 12,
415
+ "precision_mean": 0.3667929292929293,
416
+ "precision_sample_sd": 0.07670649392711647,
417
+ "recall_mean": 0.47222222222222227,
418
+ "recall_sample_sd": 0.09622504486493764,
419
+ "f1_mean": 0.4053892966936445,
420
+ "f1_sample_sd": 0.04211369218048746
421
+ },
422
+ {
423
+ "arm": "task_agnostic_base",
424
+ "label": "Causal_Oversimplification",
425
+ "support": 18,
426
+ "precision_mean": 0.5428824049513704,
427
+ "precision_sample_sd": 0.039216350192122156,
428
+ "recall_mean": 0.7962962962962963,
429
+ "recall_sample_sd": 0.11564811108145183,
430
+ "f1_mean": 0.6446992723588468,
431
+ "f1_sample_sd": 0.0634561308339175
432
+ },
433
+ {
434
+ "arm": "task_agnostic_base",
435
+ "label": "Doubt",
436
+ "support": 22,
437
+ "precision_mean": 0.5223832528180354,
438
+ "precision_sample_sd": 0.06742780050556177,
439
+ "recall_mean": 0.9848484848484849,
440
+ "recall_sample_sd": 0.026243194054073875,
441
+ "f1_mean": 0.6802057822343391,
442
+ "f1_sample_sd": 0.04928561754686331
443
+ },
444
+ {
445
+ "arm": "task_agnostic_base",
446
+ "label": "Exaggeration,Minimisation",
447
+ "support": 22,
448
+ "precision_mean": 0.4620811287477954,
449
+ "precision_sample_sd": 0.016801198309751,
450
+ "recall_mean": 0.7727272727272727,
451
+ "recall_sample_sd": 0.15745916432444337,
452
+ "f1_mean": 0.5727040816326531,
453
+ "f1_sample_sd": 0.03645259990419198
454
+ },
455
+ {
456
+ "arm": "task_agnostic_base",
457
+ "label": "Flag-Waving",
458
+ "support": 17,
459
+ "precision_mean": 0.5642642642642642,
460
+ "precision_sample_sd": 0.1478145677656334,
461
+ "recall_mean": 0.8235294117647058,
462
+ "recall_sample_sd": 0.1764705882352941,
463
+ "f1_mean": 0.6464506172839506,
464
+ "f1_sample_sd": 0.035742219952758225
465
+ },
466
+ {
467
+ "arm": "task_agnostic_base",
468
+ "label": "Loaded_Language",
469
+ "support": 45,
470
+ "precision_mean": 0.8649032907793016,
471
+ "precision_sample_sd": 0.046984312428427724,
472
+ "recall_mean": 0.9851851851851852,
473
+ "recall_sample_sd": 0.012830005981991702,
474
+ "f1_mean": 0.9208741623431562,
475
+ "f1_sample_sd": 0.03200926878737141
476
+ },
477
+ {
478
+ "arm": "task_agnostic_base",
479
+ "label": "Name_Calling,Labeling",
480
+ "support": 33,
481
+ "precision_mean": 0.7380952380952381,
482
+ "precision_sample_sd": 0.0047619047619048005,
483
+ "recall_mean": 0.9090909090909091,
484
+ "recall_sample_sd": 0.10925912955951485,
485
+ "f1_mean": 0.8125087983911513,
486
+ "f1_sample_sd": 0.04252971464057227
487
+ },
488
+ {
489
+ "arm": "task_agnostic_base",
490
+ "label": "Repetition",
491
+ "support": 23,
492
+ "precision_mean": 0.5196620583717357,
493
+ "precision_sample_sd": 0.021645909050194662,
494
+ "recall_mean": 0.7536231884057971,
495
+ "recall_sample_sd": 0.06641414050660639,
496
+ "f1_mean": 0.6147520781519571,
497
+ "f1_sample_sd": 0.03505927109416221
498
+ },
499
+ {
500
+ "arm": "task_agnostic_base",
501
+ "label": "Slogans",
502
+ "support": 14,
503
+ "precision_mean": 0.5161361626878869,
504
+ "precision_sample_sd": 0.20305677930281646,
505
+ "recall_mean": 0.7380952380952381,
506
+ "recall_sample_sd": 0.10910894511799614,
507
+ "f1_mean": 0.5834824090638044,
508
+ "f1_sample_sd": 0.09862686289220693
509
+ },
510
+ {
511
+ "arm": "task_agnostic_base",
512
+ "label": "Thought-terminating_Cliches",
513
+ "support": 7,
514
+ "precision_mean": 0.3492063492063492,
515
+ "precision_sample_sd": 0.143516994603374,
516
+ "recall_mean": 0.3333333333333333,
517
+ "recall_sample_sd": 0.08247860988423225,
518
+ "f1_mean": 0.319014319014319,
519
+ "f1_sample_sd": 0.04017590711117064
520
+ },
521
+ {
522
+ "arm": "task_agnostic_base",
523
+ "label": "Whataboutism,Straw_Men,Red_Herring",
524
+ "support": 10,
525
+ "precision_mean": 0.2761904761904762,
526
+ "precision_sample_sd": 0.05302632726504782,
527
+ "recall_mean": 0.5,
528
+ "recall_sample_sd": 0.2645751311064591,
529
+ "f1_mean": 0.330448343079922,
530
+ "f1_sample_sd": 0.021845164720973014
531
+ },
532
+ {
533
+ "arm": "structural_copy_control",
534
+ "label": "Appeal_to_Authority",
535
+ "support": 11,
536
+ "precision_mean": 0.26003086419753085,
537
+ "precision_sample_sd": 0.06281357095350094,
538
+ "recall_mean": 0.5757575757575758,
539
+ "recall_sample_sd": 0.262431940540739,
540
+ "f1_mean": 0.35477582846003897,
541
+ "f1_sample_sd": 0.11479479036518875
542
+ },
543
+ {
544
+ "arm": "structural_copy_control",
545
+ "label": "Appeal_to_fear-prejudice",
546
+ "support": 24,
547
+ "precision_mean": 0.5104813315339631,
548
+ "precision_sample_sd": 0.038427508996734555,
549
+ "recall_mean": 0.8611111111111112,
550
+ "recall_sample_sd": 0.024056261216234387,
551
+ "f1_mean": 0.6401745363877202,
552
+ "f1_sample_sd": 0.029305467347150463
553
+ },
554
+ {
555
+ "arm": "structural_copy_control",
556
+ "label": "Bandwagon,Reductio_ad_hitlerum",
557
+ "support": 7,
558
+ "precision_mean": 0.2876623376623376,
559
+ "precision_sample_sd": 0.061386823077618426,
560
+ "recall_mean": 0.8571428571428571,
561
+ "recall_sample_sd": 0.14285714285714285,
562
+ "f1_mean": 0.4306391777656145,
563
+ "f1_sample_sd": 0.08686392624022733
564
+ },
565
+ {
566
+ "arm": "structural_copy_control",
567
+ "label": "Black-and-White_Fallacy",
568
+ "support": 12,
569
+ "precision_mean": 0.31339031339031337,
570
+ "precision_sample_sd": 0.08301311842925053,
571
+ "recall_mean": 0.5277777777777778,
572
+ "recall_sample_sd": 0.2678791878053599,
573
+ "f1_mean": 0.38778427550357375,
574
+ "f1_sample_sd": 0.13069755090515112
575
+ },
576
+ {
577
+ "arm": "structural_copy_control",
578
+ "label": "Causal_Oversimplification",
579
+ "support": 18,
580
+ "precision_mean": 0.4464201043148412,
581
+ "precision_sample_sd": 0.013538446216471536,
582
+ "recall_mean": 0.9259259259259259,
583
+ "recall_sample_sd": 0.03207501495497923,
584
+ "f1_mean": 0.6023809523809524,
585
+ "f1_sample_sd": 0.018643640071557464
586
+ },
587
+ {
588
+ "arm": "structural_copy_control",
589
+ "label": "Doubt",
590
+ "support": 22,
591
+ "precision_mean": 0.4455314009661836,
592
+ "precision_sample_sd": 0.04067072694857813,
593
+ "recall_mean": 1.0,
594
+ "recall_sample_sd": 0.0,
595
+ "f1_mean": 0.615686274509804,
596
+ "f1_sample_sd": 0.039427200694482675
597
+ },
598
+ {
599
+ "arm": "structural_copy_control",
600
+ "label": "Exaggeration,Minimisation",
601
+ "support": 22,
602
+ "precision_mean": 0.4364176150420584,
603
+ "precision_sample_sd": 0.025093958106097894,
604
+ "recall_mean": 0.8787878787878788,
605
+ "recall_sample_sd": 0.05248638810814775,
606
+ "f1_mean": 0.5829803047194351,
607
+ "f1_sample_sd": 0.030899288356345502
608
+ },
609
+ {
610
+ "arm": "structural_copy_control",
611
+ "label": "Flag-Waving",
612
+ "support": 17,
613
+ "precision_mean": 0.3814814814814815,
614
+ "precision_sample_sd": 0.05010277503136551,
615
+ "recall_mean": 0.803921568627451,
616
+ "recall_sample_sd": 0.17970885078258195,
617
+ "f1_mean": 0.5138595287119652,
618
+ "f1_sample_sd": 0.06400931989326974
619
+ },
620
+ {
621
+ "arm": "structural_copy_control",
622
+ "label": "Loaded_Language",
623
+ "support": 45,
624
+ "precision_mean": 0.8492581841638446,
625
+ "precision_sample_sd": 0.016026591848331686,
626
+ "recall_mean": 1.0,
627
+ "recall_sample_sd": 0.0,
628
+ "f1_mean": 0.9184311025253588,
629
+ "f1_sample_sd": 0.009372233868132662
630
+ },
631
+ {
632
+ "arm": "structural_copy_control",
633
+ "label": "Name_Calling,Labeling",
634
+ "support": 33,
635
+ "precision_mean": 0.6375,
636
+ "precision_sample_sd": 0.03410563654946854,
637
+ "recall_mean": 0.9696969696969697,
638
+ "recall_sample_sd": 0.030303030303030276,
639
+ "f1_mean": 0.7685185185185185,
640
+ "f1_sample_sd": 0.02023900779105559
641
+ },
642
+ {
643
+ "arm": "structural_copy_control",
644
+ "label": "Repetition",
645
+ "support": 23,
646
+ "precision_mean": 0.44863933452168747,
647
+ "precision_sample_sd": 0.01520014564765896,
648
+ "recall_mean": 0.9420289855072463,
649
+ "recall_sample_sd": 0.06641414050660639,
650
+ "f1_mean": 0.607248835423033,
651
+ "f1_sample_sd": 0.019858627551810427
652
+ },
653
+ {
654
+ "arm": "structural_copy_control",
655
+ "label": "Slogans",
656
+ "support": 14,
657
+ "precision_mean": 0.3274318274318274,
658
+ "precision_sample_sd": 0.05434598572773684,
659
+ "recall_mean": 0.4523809523809524,
660
+ "recall_sample_sd": 0.1486904285332952,
661
+ "f1_mean": 0.3715828924162258,
662
+ "f1_sample_sd": 0.07106660236343577
663
+ },
664
+ {
665
+ "arm": "structural_copy_control",
666
+ "label": "Thought-terminating_Cliches",
667
+ "support": 7,
668
+ "precision_mean": 0.1142156862745098,
669
+ "precision_sample_sd": 0.012848374923608973,
670
+ "recall_mean": 0.38095238095238093,
671
+ "recall_sample_sd": 0.1649572197684645,
672
+ "f1_mean": 0.17331433998100665,
673
+ "f1_sample_sd": 0.029065880233243847
674
+ },
675
+ {
676
+ "arm": "structural_copy_control",
677
+ "label": "Whataboutism,Straw_Men,Red_Herring",
678
+ "support": 10,
679
+ "precision_mean": 0.19344598054275472,
680
+ "precision_sample_sd": 0.03373872471827226,
681
+ "recall_mean": 0.5666666666666667,
682
+ "recall_sample_sd": 0.2081665999466133,
683
+ "f1_mean": 0.28584083690299267,
684
+ "f1_sample_sd": 0.06078854830133676
685
+ },
686
+ {
687
+ "arm": "existing_specialized_separate_lineage",
688
+ "label": "Appeal_to_Authority",
689
+ "support": 11,
690
+ "precision_mean": 0.37836021505376344,
691
+ "precision_sample_sd": 0.10546365636105155,
692
+ "recall_mean": 0.6666666666666666,
693
+ "recall_sample_sd": 0.2287828616748712,
694
+ "f1_mean": 0.46885361552028215,
695
+ "f1_sample_sd": 0.09502747629890008
696
+ },
697
+ {
698
+ "arm": "existing_specialized_separate_lineage",
699
+ "label": "Appeal_to_fear-prejudice",
700
+ "support": 24,
701
+ "precision_mean": 0.525708635464733,
702
+ "precision_sample_sd": 0.02235198659615819,
703
+ "recall_mean": 0.8888888888888888,
704
+ "recall_sample_sd": 0.048112522432468774,
705
+ "f1_mean": 0.6599065351235072,
706
+ "f1_sample_sd": 0.015362381542320084
707
+ },
708
+ {
709
+ "arm": "existing_specialized_separate_lineage",
710
+ "label": "Bandwagon,Reductio_ad_hitlerum",
711
+ "support": 7,
712
+ "precision_mean": 0.44485144485144484,
713
+ "precision_sample_sd": 0.016487284575115283,
714
+ "recall_mean": 0.7619047619047619,
715
+ "recall_sample_sd": 0.1649572197684645,
716
+ "f1_mean": 0.5571428571428572,
717
+ "f1_sample_sd": 0.051507875363771265
718
+ },
719
+ {
720
+ "arm": "existing_specialized_separate_lineage",
721
+ "label": "Black-and-White_Fallacy",
722
+ "support": 12,
723
+ "precision_mean": 0.4662644544997486,
724
+ "precision_sample_sd": 0.004538274146363688,
725
+ "recall_mean": 0.5833333333333334,
726
+ "recall_sample_sd": 0.08333333333333331,
727
+ "f1_mean": 0.516747552149851,
728
+ "f1_sample_sd": 0.03589484965697345
729
+ },
730
+ {
731
+ "arm": "existing_specialized_separate_lineage",
732
+ "label": "Causal_Oversimplification",
733
+ "support": 18,
734
+ "precision_mean": 0.4472934472934473,
735
+ "precision_sample_sd": 0.013055771210700413,
736
+ "recall_mean": 0.8333333333333333,
737
+ "recall_sample_sd": 0.1469861839480328,
738
+ "f1_mean": 0.5781794553724379,
739
+ "f1_sample_sd": 0.028407563015813122
740
+ },
741
+ {
742
+ "arm": "existing_specialized_separate_lineage",
743
+ "label": "Doubt",
744
+ "support": 22,
745
+ "precision_mean": 0.5045661001171619,
746
+ "precision_sample_sd": 0.023581174969843577,
747
+ "recall_mean": 1.0,
748
+ "recall_sample_sd": 0.0,
749
+ "f1_mean": 0.6704939668174963,
750
+ "f1_sample_sd": 0.020973132363659372
751
+ },
752
+ {
753
+ "arm": "existing_specialized_separate_lineage",
754
+ "label": "Exaggeration,Minimisation",
755
+ "support": 22,
756
+ "precision_mean": 0.44589793281653745,
757
+ "precision_sample_sd": 0.010987847993006481,
758
+ "recall_mean": 0.9393939393939394,
759
+ "recall_sample_sd": 0.06943296507508849,
760
+ "f1_mean": 0.6043956043956044,
761
+ "f1_sample_sd": 0.022305256187015846
762
+ },
763
+ {
764
+ "arm": "existing_specialized_separate_lineage",
765
+ "label": "Flag-Waving",
766
+ "support": 17,
767
+ "precision_mean": 0.5576388888888889,
768
+ "precision_sample_sd": 0.03705367103320583,
769
+ "recall_mean": 0.8823529411764706,
770
+ "recall_sample_sd": 0.11764705882352944,
771
+ "f1_mean": 0.680769868923179,
772
+ "f1_sample_sd": 0.04164658769406446
773
+ },
774
+ {
775
+ "arm": "existing_specialized_separate_lineage",
776
+ "label": "Loaded_Language",
777
+ "support": 45,
778
+ "precision_mean": 0.8387650085763294,
779
+ "precision_sample_sd": 0.01782556577257849,
780
+ "recall_mean": 1.0,
781
+ "recall_sample_sd": 0.0,
782
+ "f1_mean": 0.9122448979591837,
783
+ "f1_sample_sd": 0.01060439269940129
784
+ },
785
+ {
786
+ "arm": "existing_specialized_separate_lineage",
787
+ "label": "Name_Calling,Labeling",
788
+ "support": 33,
789
+ "precision_mean": 0.6957104190221673,
790
+ "precision_sample_sd": 0.019201490809269257,
791
+ "recall_mean": 0.9696969696969697,
792
+ "recall_sample_sd": 0.030303030303030276,
793
+ "f1_mean": 0.8101049442821594,
794
+ "f1_sample_sd": 0.022092733328802537
795
+ },
796
+ {
797
+ "arm": "existing_specialized_separate_lineage",
798
+ "label": "Repetition",
799
+ "support": 23,
800
+ "precision_mean": 0.4781557067271353,
801
+ "precision_sample_sd": 0.009897987322673213,
802
+ "recall_mean": 0.9420289855072463,
803
+ "recall_sample_sd": 0.06641414050660639,
804
+ "f1_mean": 0.6337774426009719,
805
+ "f1_sample_sd": 0.016444118333485254
806
+ },
807
+ {
808
+ "arm": "existing_specialized_separate_lineage",
809
+ "label": "Slogans",
810
+ "support": 14,
811
+ "precision_mean": 0.4334215167548501,
812
+ "precision_sample_sd": 0.05811581861282867,
813
+ "recall_mean": 0.6904761904761905,
814
+ "recall_sample_sd": 0.1649572197684645,
815
+ "f1_mean": 0.5201316298877274,
816
+ "f1_sample_sd": 0.018567913895656114
817
+ },
818
+ {
819
+ "arm": "existing_specialized_separate_lineage",
820
+ "label": "Thought-terminating_Cliches",
821
+ "support": 7,
822
+ "precision_mean": 0.18446115288220552,
823
+ "precision_sample_sd": 0.03641251252864963,
824
+ "recall_mean": 0.6666666666666666,
825
+ "recall_sample_sd": 0.1649572197684645,
826
+ "f1_mean": 0.2868626868626869,
827
+ "f1_sample_sd": 0.0511620864429468
828
+ },
829
+ {
830
+ "arm": "existing_specialized_separate_lineage",
831
+ "label": "Whataboutism,Straw_Men,Red_Herring",
832
+ "support": 10,
833
+ "precision_mean": 0.2777777777777778,
834
+ "precision_sample_sd": 0.0481125224324688,
835
+ "recall_mean": 0.4,
836
+ "recall_sample_sd": 0.0,
837
+ "f1_mean": 0.32634032634032634,
838
+ "f1_sample_sd": 0.03229931575886014
839
+ }
840
+ ],
841
+ "compression_ladder": [
842
+ {
843
+ "name": "24L upstream text teacher",
844
+ "layers": 24,
845
+ "parameters": 752393024,
846
+ "weight_bytes": 1504825632,
847
+ "weight_mib": 1435.1135559082031,
848
+ "weight_sha256": "30783da4651259c26b8a204821ea426b9526431e0c14da0d1c6fa284e7414e5e",
849
+ "kd_stage": "",
850
+ "mean_total_loss": "",
851
+ "mean_interface_loss": "",
852
+ "mean_final_loss": ""
853
+ },
854
+ {
855
+ "name": "8L task-free KD",
856
+ "layers": 8,
857
+ "parameters": 420318144,
858
+ "weight_bytes": 840647880,
859
+ "weight_mib": 801.7042922973633,
860
+ "weight_sha256": "086fbaf9a4838ecde56cd3541c9c3212b22d0c5ee745cf2c1c5e0bb31e0ec22c",
861
+ "kd_stage": "24to8",
862
+ "mean_total_loss": 0.17622653172702485,
863
+ "mean_interface_loss": 0.21562811763578793,
864
+ "mean_final_loss": 0.16626427527853593
865
+ },
866
+ {
867
+ "name": "6L task-free KD",
868
+ "layers": 6,
869
+ "parameters": 377207424,
870
+ "weight_bytes": 754423424,
871
+ "weight_mib": 719.4742431640625,
872
+ "weight_sha256": "492749c20f77a42f6bc993f0927a82217b94f872b7eacf25612f7520c9bc28ce",
873
+ "kd_stage": "8to6",
874
+ "mean_total_loss": 0.03382104352249371,
875
+ "mean_interface_loss": 0.04486069045515251,
876
+ "mean_final_loss": 0.031147060785087888
877
+ },
878
+ {
879
+ "name": "4L task-free KD root",
880
+ "layers": 4,
881
+ "parameters": 334096704,
882
+ "weight_bytes": 668198976,
883
+ "weight_mib": 637.2442016601562,
884
+ "weight_sha256": "2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3",
885
+ "kd_stage": "6to4",
886
+ "mean_total_loss": 0.047020394468859195,
887
+ "mean_interface_loss": 0.07048421185527332,
888
+ "mean_final_loss": 0.043119939065377366
889
+ }
890
+ ]
891
  }
benchmark/summary.json ADDED
@@ -0,0 +1,891 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "schema": "standalone4l-complete-transfer-benchmark-v2",
3
+ "status": "complete_from_frozen_reports",
4
+ "claim_scope": "Exploratory fresh-head transfer evidence on one English task and a previously opened 55-article SemEval-derived test split.",
5
+ "headless_root_requires_adaptation": true,
6
+ "test_previously_opened": true,
7
+ "test_articles": 55,
8
+ "test_windows": 434,
9
+ "seeds": [
10
+ 41,
11
+ 42,
12
+ 43
13
+ ],
14
+ "three_seed_sd_is_not_confidence_interval": true,
15
+ "multilingual_downstream_evaluation_performed": false,
16
+ "upstream_multilingual_metadata": {
17
+ "source": "Qwen/Qwen3.5-0.8B official model card",
18
+ "languages_and_dialects_claimed_upstream": 201,
19
+ "scope": "inherited tokenizer and architecture coverage only"
20
+ },
21
+ "protocol": {
22
+ "epochs": 5,
23
+ "primary_epoch": 5,
24
+ "early_stopping": false,
25
+ "max_length": 256,
26
+ "stride": 128,
27
+ "window_target": "span-aligned technique labels",
28
+ "article_aggregation": "labelwise maximum probability",
29
+ "micro_batch_size": 1,
30
+ "gradient_accumulation": 32,
31
+ "effective_batch_size": 32,
32
+ "learning_rate": 2e-05,
33
+ "weight_decay": 0.01,
34
+ "gradient_clip": 1.0,
35
+ "optimizer": "AdamW",
36
+ "dtype": "bfloat16",
37
+ "thresholds": [
38
+ 0.05,
39
+ 0.1,
40
+ 0.15,
41
+ 0.2,
42
+ 0.25,
43
+ 0.3,
44
+ 0.35,
45
+ 0.4,
46
+ 0.45,
47
+ 0.5,
48
+ 0.55,
49
+ 0.6,
50
+ 0.65,
51
+ 0.7,
52
+ 0.75,
53
+ 0.8,
54
+ 0.85,
55
+ 0.9
56
+ ],
57
+ "threshold_selection": "epoch-5 calibration macro-F1, then micro-F1"
58
+ },
59
+ "arms": {
60
+ "task_agnostic_base": {
61
+ "display": "Task-agnostic KD base",
62
+ "lineage": "root base plus a fresh 14-label transfer head",
63
+ "metrics": {
64
+ "macro_f1": {
65
+ "values": [
66
+ 0.5883093293517536,
67
+ 0.6028379036218096,
68
+ 0.5712816197954101
69
+ ],
70
+ "mean": 0.5874762842563245,
71
+ "sample_sd": 0.01579462678670608,
72
+ "n": 3
73
+ },
74
+ "micro_f1": {
75
+ "values": [
76
+ 0.6468085106382978,
77
+ 0.6655172413793103,
78
+ 0.6465256797583082
79
+ ],
80
+ "mean": 0.6529504772586388,
81
+ "sample_sd": 0.010884055708931178,
82
+ "n": 3
83
+ },
84
+ "exact_match": {
85
+ "values": [
86
+ 0.05454545454545454,
87
+ 0.03636363636363636,
88
+ 0.0
89
+ ],
90
+ "mean": 0.0303030303030303,
91
+ "sample_sd": 0.027773186030035393,
92
+ "n": 3
93
+ },
94
+ "predicted_positive_rate": {
95
+ "values": [
96
+ 0.5714285714285714,
97
+ 0.4090909090909091,
98
+ 0.5155844155844156
99
+ ],
100
+ "mean": 0.4987012987012987,
101
+ "sample_sd": 0.08247520160679181,
102
+ "n": 3
103
+ }
104
+ },
105
+ "resources": {
106
+ "parameters": {
107
+ "values": [
108
+ 334111040.0,
109
+ 334111040.0,
110
+ 334111040.0
111
+ ],
112
+ "mean": 334111040.0,
113
+ "sample_sd": 0.0,
114
+ "n": 3
115
+ },
116
+ "peak_cuda_allocated_bytes": {
117
+ "values": [
118
+ 3380975616.0,
119
+ 3380975616.0,
120
+ 3380975616.0
121
+ ],
122
+ "mean": 3380975616.0,
123
+ "sample_sd": 0.0,
124
+ "n": 3
125
+ },
126
+ "peak_cuda_reserved_bytes": {
127
+ "values": [
128
+ 3519021056.0,
129
+ 3519021056.0,
130
+ 3510632448.0
131
+ ],
132
+ "mean": 3516224853.3333335,
133
+ "sample_sd": 4843165.086926248,
134
+ "n": 3
135
+ },
136
+ "total_wall_seconds": {
137
+ "values": [
138
+ 1583.0297977919981,
139
+ 1588.617994428001,
140
+ 1603.4310500120046
141
+ ],
142
+ "mean": 1591.692947410668,
143
+ "sample_sd": 10.54249851412311,
144
+ "n": 3
145
+ },
146
+ "test_seconds_per_article": {
147
+ "values": [
148
+ 0.08164824154545468,
149
+ 0.07871871667289683,
150
+ 0.08355205721825107
151
+ ],
152
+ "mean": 0.08130633847886752,
153
+ "sample_sd": 0.0024347419754560395,
154
+ "n": 3
155
+ }
156
+ },
157
+ "thresholds": [
158
+ 0.15,
159
+ 0.3,
160
+ 0.1
161
+ ]
162
+ },
163
+ "structural_copy_control": {
164
+ "display": "Structural copy, no task-free KD",
165
+ "lineage": "same four-layer structure without task-free KD",
166
+ "metrics": {
167
+ "macro_f1": {
168
+ "values": [
169
+ 0.5169424579262757,
170
+ 0.5349405779540175,
171
+ 0.5023778364496152
172
+ ],
173
+ "mean": 0.5180869574433028,
174
+ "sample_sd": 0.016311512588628545,
175
+ "n": 3
176
+ },
177
+ "micro_f1": {
178
+ "values": [
179
+ 0.5773955773955775,
180
+ 0.6083445491251682,
181
+ 0.585635359116022
182
+ ],
183
+ "mean": 0.5904584952122559,
184
+ "sample_sd": 0.01602830914762526,
185
+ "n": 3
186
+ },
187
+ "exact_match": {
188
+ "values": [
189
+ 0.0,
190
+ 0.0,
191
+ 0.0
192
+ ],
193
+ "mean": 0.0,
194
+ "sample_sd": 0.0,
195
+ "n": 3
196
+ },
197
+ "predicted_positive_rate": {
198
+ "values": [
199
+ 0.712987012987013,
200
+ 0.6207792207792208,
201
+ 0.5961038961038961
202
+ ],
203
+ "mean": 0.6432900432900432,
204
+ "sample_sd": 0.06160737456395461,
205
+ "n": 3
206
+ }
207
+ },
208
+ "resources": {
209
+ "parameters": {
210
+ "values": [
211
+ 334111040.0,
212
+ 334111040.0,
213
+ 334111040.0
214
+ ],
215
+ "mean": 334111040.0,
216
+ "sample_sd": 0.0,
217
+ "n": 3
218
+ },
219
+ "peak_cuda_allocated_bytes": {
220
+ "values": [
221
+ 3380975616.0,
222
+ 3380975616.0,
223
+ 3380975616.0
224
+ ],
225
+ "mean": 3380975616.0,
226
+ "sample_sd": 0.0,
227
+ "n": 3
228
+ },
229
+ "peak_cuda_reserved_bytes": {
230
+ "values": [
231
+ 3519021056.0,
232
+ 3519021056.0,
233
+ 3510632448.0
234
+ ],
235
+ "mean": 3516224853.3333335,
236
+ "sample_sd": 4843165.086926248,
237
+ "n": 3
238
+ },
239
+ "total_wall_seconds": {
240
+ "values": [
241
+ 1575.5905925639963,
242
+ 1623.5639530190092,
243
+ 1629.195114912989
244
+ ],
245
+ "mean": 1609.4498868319981,
246
+ "sample_sd": 29.457874198835384,
247
+ "n": 3
248
+ },
249
+ "test_seconds_per_article": {
250
+ "values": [
251
+ 0.07877608058201573,
252
+ 0.08307423260041767,
253
+ 0.08163602005436339
254
+ ],
255
+ "mean": 0.08116211107893226,
256
+ "sample_sd": 0.002187914527825257,
257
+ "n": 3
258
+ }
259
+ },
260
+ "thresholds": [
261
+ 0.05,
262
+ 0.2,
263
+ 0.2
264
+ ]
265
+ },
266
+ "existing_specialized_separate_lineage": {
267
+ "display": "Existing specialized 4L, separate lineage",
268
+ "lineage": "historical task-specific shrink/distillation lineage",
269
+ "metrics": {
270
+ "macro_f1": {
271
+ "values": [
272
+ 0.5832848452288824,
273
+ 0.5958405181160762,
274
+ 0.5835785045218137
275
+ ],
276
+ "mean": 0.5875679559555907,
277
+ "sample_sd": 0.007165753447433348,
278
+ "n": 3
279
+ },
280
+ "micro_f1": {
281
+ "values": [
282
+ 0.6494252873563218,
283
+ 0.6458616010854816,
284
+ 0.6486486486486486
285
+ ],
286
+ "mean": 0.647978512363484,
287
+ "sample_sd": 0.0018739735696546695,
288
+ "n": 3
289
+ },
290
+ "exact_match": {
291
+ "values": [
292
+ 0.0,
293
+ 0.0,
294
+ 0.01818181818181818
295
+ ],
296
+ "mean": 0.006060606060606061,
297
+ "sample_sd": 0.01049727762162956,
298
+ "n": 3
299
+ },
300
+ "predicted_positive_rate": {
301
+ "values": [
302
+ 0.5597402597402598,
303
+ 0.612987012987013,
304
+ 0.5688311688311688
305
+ ],
306
+ "mean": 0.5805194805194805,
307
+ "sample_sd": 0.02848274311618351,
308
+ "n": 3
309
+ }
310
+ },
311
+ "resources": {
312
+ "parameters": {
313
+ "values": [
314
+ 210903360.0,
315
+ 210903360.0,
316
+ 210903360.0
317
+ ],
318
+ "mean": 210903360.0,
319
+ "sample_sd": 0.0,
320
+ "n": 3
321
+ },
322
+ "peak_cuda_allocated_bytes": {
323
+ "values": [
324
+ 2143655936.0,
325
+ 2143655936.0,
326
+ 2143655936.0
327
+ ],
328
+ "mean": 2143655936.0,
329
+ "sample_sd": 0.0,
330
+ "n": 3
331
+ },
332
+ "peak_cuda_reserved_bytes": {
333
+ "values": [
334
+ 2281701376.0,
335
+ 2281701376.0,
336
+ 2273312768.0
337
+ ],
338
+ "mean": 2278905173.3333335,
339
+ "sample_sd": 4843165.086926248,
340
+ "n": 3
341
+ },
342
+ "total_wall_seconds": {
343
+ "values": [
344
+ 1634.488557530014,
345
+ 1652.9397718419787,
346
+ 1643.488202479988
347
+ ],
348
+ "mean": 1643.6388439506602,
349
+ "sample_sd": 9.226529522850198,
350
+ "n": 3
351
+ },
352
+ "test_seconds_per_article": {
353
+ "values": [
354
+ 0.0847959352360869,
355
+ 0.08598086021798239,
356
+ 0.08552987641781908
357
+ ],
358
+ "mean": 0.08543555729062946,
359
+ "sample_sd": 0.0005980667826383295,
360
+ "n": 3
361
+ }
362
+ },
363
+ "thresholds": [
364
+ 0.1,
365
+ 0.05,
366
+ 0.05
367
+ ]
368
+ }
369
+ },
370
+ "same_seed_macro_f1_deltas_base_minus_control": [
371
+ 0.07136687142547793,
372
+ 0.06789732566779205,
373
+ 0.06890378334579494
374
+ ],
375
+ "mean_macro_f1_delta_base_minus_control": 0.06938932681302164,
376
+ "publication_gate_passed": true,
377
+ "per_label_summary": [
378
+ {
379
+ "arm": "task_agnostic_base",
380
+ "label": "Appeal_to_Authority",
381
+ "support": 11,
382
+ "precision_mean": 0.3958614484930274,
383
+ "precision_sample_sd": 0.1238064457010119,
384
+ "recall_mean": 0.5757575757575757,
385
+ "recall_sample_sd": 0.05248638810814781,
386
+ "f1_mean": 0.4657088122605364,
387
+ "f1_sample_sd": 0.1020990126979069
388
+ },
389
+ {
390
+ "arm": "task_agnostic_base",
391
+ "label": "Appeal_to_fear-prejudice",
392
+ "support": 24,
393
+ "precision_mean": 0.5540244550770866,
394
+ "precision_sample_sd": 0.02525673990402998,
395
+ "recall_mean": 0.7361111111111112,
396
+ "recall_sample_sd": 0.13393959390267993,
397
+ "f1_mean": 0.6277521761392729,
398
+ "f1_sample_sd": 0.04994879016183189
399
+ },
400
+ {
401
+ "arm": "task_agnostic_base",
402
+ "label": "Bandwagon,Reductio_ad_hitlerum",
403
+ "support": 7,
404
+ "precision_mean": 0.4851851851851852,
405
+ "precision_sample_sd": 0.07882887649552875,
406
+ "recall_mean": 0.8095238095238095,
407
+ "recall_sample_sd": 0.08247860988423221,
408
+ "f1_mean": 0.6006778309409889,
409
+ "f1_sample_sd": 0.04793776258918466
410
+ },
411
+ {
412
+ "arm": "task_agnostic_base",
413
+ "label": "Black-and-White_Fallacy",
414
+ "support": 12,
415
+ "precision_mean": 0.3667929292929293,
416
+ "precision_sample_sd": 0.07670649392711647,
417
+ "recall_mean": 0.47222222222222227,
418
+ "recall_sample_sd": 0.09622504486493764,
419
+ "f1_mean": 0.4053892966936445,
420
+ "f1_sample_sd": 0.04211369218048746
421
+ },
422
+ {
423
+ "arm": "task_agnostic_base",
424
+ "label": "Causal_Oversimplification",
425
+ "support": 18,
426
+ "precision_mean": 0.5428824049513704,
427
+ "precision_sample_sd": 0.039216350192122156,
428
+ "recall_mean": 0.7962962962962963,
429
+ "recall_sample_sd": 0.11564811108145183,
430
+ "f1_mean": 0.6446992723588468,
431
+ "f1_sample_sd": 0.0634561308339175
432
+ },
433
+ {
434
+ "arm": "task_agnostic_base",
435
+ "label": "Doubt",
436
+ "support": 22,
437
+ "precision_mean": 0.5223832528180354,
438
+ "precision_sample_sd": 0.06742780050556177,
439
+ "recall_mean": 0.9848484848484849,
440
+ "recall_sample_sd": 0.026243194054073875,
441
+ "f1_mean": 0.6802057822343391,
442
+ "f1_sample_sd": 0.04928561754686331
443
+ },
444
+ {
445
+ "arm": "task_agnostic_base",
446
+ "label": "Exaggeration,Minimisation",
447
+ "support": 22,
448
+ "precision_mean": 0.4620811287477954,
449
+ "precision_sample_sd": 0.016801198309751,
450
+ "recall_mean": 0.7727272727272727,
451
+ "recall_sample_sd": 0.15745916432444337,
452
+ "f1_mean": 0.5727040816326531,
453
+ "f1_sample_sd": 0.03645259990419198
454
+ },
455
+ {
456
+ "arm": "task_agnostic_base",
457
+ "label": "Flag-Waving",
458
+ "support": 17,
459
+ "precision_mean": 0.5642642642642642,
460
+ "precision_sample_sd": 0.1478145677656334,
461
+ "recall_mean": 0.8235294117647058,
462
+ "recall_sample_sd": 0.1764705882352941,
463
+ "f1_mean": 0.6464506172839506,
464
+ "f1_sample_sd": 0.035742219952758225
465
+ },
466
+ {
467
+ "arm": "task_agnostic_base",
468
+ "label": "Loaded_Language",
469
+ "support": 45,
470
+ "precision_mean": 0.8649032907793016,
471
+ "precision_sample_sd": 0.046984312428427724,
472
+ "recall_mean": 0.9851851851851852,
473
+ "recall_sample_sd": 0.012830005981991702,
474
+ "f1_mean": 0.9208741623431562,
475
+ "f1_sample_sd": 0.03200926878737141
476
+ },
477
+ {
478
+ "arm": "task_agnostic_base",
479
+ "label": "Name_Calling,Labeling",
480
+ "support": 33,
481
+ "precision_mean": 0.7380952380952381,
482
+ "precision_sample_sd": 0.0047619047619048005,
483
+ "recall_mean": 0.9090909090909091,
484
+ "recall_sample_sd": 0.10925912955951485,
485
+ "f1_mean": 0.8125087983911513,
486
+ "f1_sample_sd": 0.04252971464057227
487
+ },
488
+ {
489
+ "arm": "task_agnostic_base",
490
+ "label": "Repetition",
491
+ "support": 23,
492
+ "precision_mean": 0.5196620583717357,
493
+ "precision_sample_sd": 0.021645909050194662,
494
+ "recall_mean": 0.7536231884057971,
495
+ "recall_sample_sd": 0.06641414050660639,
496
+ "f1_mean": 0.6147520781519571,
497
+ "f1_sample_sd": 0.03505927109416221
498
+ },
499
+ {
500
+ "arm": "task_agnostic_base",
501
+ "label": "Slogans",
502
+ "support": 14,
503
+ "precision_mean": 0.5161361626878869,
504
+ "precision_sample_sd": 0.20305677930281646,
505
+ "recall_mean": 0.7380952380952381,
506
+ "recall_sample_sd": 0.10910894511799614,
507
+ "f1_mean": 0.5834824090638044,
508
+ "f1_sample_sd": 0.09862686289220693
509
+ },
510
+ {
511
+ "arm": "task_agnostic_base",
512
+ "label": "Thought-terminating_Cliches",
513
+ "support": 7,
514
+ "precision_mean": 0.3492063492063492,
515
+ "precision_sample_sd": 0.143516994603374,
516
+ "recall_mean": 0.3333333333333333,
517
+ "recall_sample_sd": 0.08247860988423225,
518
+ "f1_mean": 0.319014319014319,
519
+ "f1_sample_sd": 0.04017590711117064
520
+ },
521
+ {
522
+ "arm": "task_agnostic_base",
523
+ "label": "Whataboutism,Straw_Men,Red_Herring",
524
+ "support": 10,
525
+ "precision_mean": 0.2761904761904762,
526
+ "precision_sample_sd": 0.05302632726504782,
527
+ "recall_mean": 0.5,
528
+ "recall_sample_sd": 0.2645751311064591,
529
+ "f1_mean": 0.330448343079922,
530
+ "f1_sample_sd": 0.021845164720973014
531
+ },
532
+ {
533
+ "arm": "structural_copy_control",
534
+ "label": "Appeal_to_Authority",
535
+ "support": 11,
536
+ "precision_mean": 0.26003086419753085,
537
+ "precision_sample_sd": 0.06281357095350094,
538
+ "recall_mean": 0.5757575757575758,
539
+ "recall_sample_sd": 0.262431940540739,
540
+ "f1_mean": 0.35477582846003897,
541
+ "f1_sample_sd": 0.11479479036518875
542
+ },
543
+ {
544
+ "arm": "structural_copy_control",
545
+ "label": "Appeal_to_fear-prejudice",
546
+ "support": 24,
547
+ "precision_mean": 0.5104813315339631,
548
+ "precision_sample_sd": 0.038427508996734555,
549
+ "recall_mean": 0.8611111111111112,
550
+ "recall_sample_sd": 0.024056261216234387,
551
+ "f1_mean": 0.6401745363877202,
552
+ "f1_sample_sd": 0.029305467347150463
553
+ },
554
+ {
555
+ "arm": "structural_copy_control",
556
+ "label": "Bandwagon,Reductio_ad_hitlerum",
557
+ "support": 7,
558
+ "precision_mean": 0.2876623376623376,
559
+ "precision_sample_sd": 0.061386823077618426,
560
+ "recall_mean": 0.8571428571428571,
561
+ "recall_sample_sd": 0.14285714285714285,
562
+ "f1_mean": 0.4306391777656145,
563
+ "f1_sample_sd": 0.08686392624022733
564
+ },
565
+ {
566
+ "arm": "structural_copy_control",
567
+ "label": "Black-and-White_Fallacy",
568
+ "support": 12,
569
+ "precision_mean": 0.31339031339031337,
570
+ "precision_sample_sd": 0.08301311842925053,
571
+ "recall_mean": 0.5277777777777778,
572
+ "recall_sample_sd": 0.2678791878053599,
573
+ "f1_mean": 0.38778427550357375,
574
+ "f1_sample_sd": 0.13069755090515112
575
+ },
576
+ {
577
+ "arm": "structural_copy_control",
578
+ "label": "Causal_Oversimplification",
579
+ "support": 18,
580
+ "precision_mean": 0.4464201043148412,
581
+ "precision_sample_sd": 0.013538446216471536,
582
+ "recall_mean": 0.9259259259259259,
583
+ "recall_sample_sd": 0.03207501495497923,
584
+ "f1_mean": 0.6023809523809524,
585
+ "f1_sample_sd": 0.018643640071557464
586
+ },
587
+ {
588
+ "arm": "structural_copy_control",
589
+ "label": "Doubt",
590
+ "support": 22,
591
+ "precision_mean": 0.4455314009661836,
592
+ "precision_sample_sd": 0.04067072694857813,
593
+ "recall_mean": 1.0,
594
+ "recall_sample_sd": 0.0,
595
+ "f1_mean": 0.615686274509804,
596
+ "f1_sample_sd": 0.039427200694482675
597
+ },
598
+ {
599
+ "arm": "structural_copy_control",
600
+ "label": "Exaggeration,Minimisation",
601
+ "support": 22,
602
+ "precision_mean": 0.4364176150420584,
603
+ "precision_sample_sd": 0.025093958106097894,
604
+ "recall_mean": 0.8787878787878788,
605
+ "recall_sample_sd": 0.05248638810814775,
606
+ "f1_mean": 0.5829803047194351,
607
+ "f1_sample_sd": 0.030899288356345502
608
+ },
609
+ {
610
+ "arm": "structural_copy_control",
611
+ "label": "Flag-Waving",
612
+ "support": 17,
613
+ "precision_mean": 0.3814814814814815,
614
+ "precision_sample_sd": 0.05010277503136551,
615
+ "recall_mean": 0.803921568627451,
616
+ "recall_sample_sd": 0.17970885078258195,
617
+ "f1_mean": 0.5138595287119652,
618
+ "f1_sample_sd": 0.06400931989326974
619
+ },
620
+ {
621
+ "arm": "structural_copy_control",
622
+ "label": "Loaded_Language",
623
+ "support": 45,
624
+ "precision_mean": 0.8492581841638446,
625
+ "precision_sample_sd": 0.016026591848331686,
626
+ "recall_mean": 1.0,
627
+ "recall_sample_sd": 0.0,
628
+ "f1_mean": 0.9184311025253588,
629
+ "f1_sample_sd": 0.009372233868132662
630
+ },
631
+ {
632
+ "arm": "structural_copy_control",
633
+ "label": "Name_Calling,Labeling",
634
+ "support": 33,
635
+ "precision_mean": 0.6375,
636
+ "precision_sample_sd": 0.03410563654946854,
637
+ "recall_mean": 0.9696969696969697,
638
+ "recall_sample_sd": 0.030303030303030276,
639
+ "f1_mean": 0.7685185185185185,
640
+ "f1_sample_sd": 0.02023900779105559
641
+ },
642
+ {
643
+ "arm": "structural_copy_control",
644
+ "label": "Repetition",
645
+ "support": 23,
646
+ "precision_mean": 0.44863933452168747,
647
+ "precision_sample_sd": 0.01520014564765896,
648
+ "recall_mean": 0.9420289855072463,
649
+ "recall_sample_sd": 0.06641414050660639,
650
+ "f1_mean": 0.607248835423033,
651
+ "f1_sample_sd": 0.019858627551810427
652
+ },
653
+ {
654
+ "arm": "structural_copy_control",
655
+ "label": "Slogans",
656
+ "support": 14,
657
+ "precision_mean": 0.3274318274318274,
658
+ "precision_sample_sd": 0.05434598572773684,
659
+ "recall_mean": 0.4523809523809524,
660
+ "recall_sample_sd": 0.1486904285332952,
661
+ "f1_mean": 0.3715828924162258,
662
+ "f1_sample_sd": 0.07106660236343577
663
+ },
664
+ {
665
+ "arm": "structural_copy_control",
666
+ "label": "Thought-terminating_Cliches",
667
+ "support": 7,
668
+ "precision_mean": 0.1142156862745098,
669
+ "precision_sample_sd": 0.012848374923608973,
670
+ "recall_mean": 0.38095238095238093,
671
+ "recall_sample_sd": 0.1649572197684645,
672
+ "f1_mean": 0.17331433998100665,
673
+ "f1_sample_sd": 0.029065880233243847
674
+ },
675
+ {
676
+ "arm": "structural_copy_control",
677
+ "label": "Whataboutism,Straw_Men,Red_Herring",
678
+ "support": 10,
679
+ "precision_mean": 0.19344598054275472,
680
+ "precision_sample_sd": 0.03373872471827226,
681
+ "recall_mean": 0.5666666666666667,
682
+ "recall_sample_sd": 0.2081665999466133,
683
+ "f1_mean": 0.28584083690299267,
684
+ "f1_sample_sd": 0.06078854830133676
685
+ },
686
+ {
687
+ "arm": "existing_specialized_separate_lineage",
688
+ "label": "Appeal_to_Authority",
689
+ "support": 11,
690
+ "precision_mean": 0.37836021505376344,
691
+ "precision_sample_sd": 0.10546365636105155,
692
+ "recall_mean": 0.6666666666666666,
693
+ "recall_sample_sd": 0.2287828616748712,
694
+ "f1_mean": 0.46885361552028215,
695
+ "f1_sample_sd": 0.09502747629890008
696
+ },
697
+ {
698
+ "arm": "existing_specialized_separate_lineage",
699
+ "label": "Appeal_to_fear-prejudice",
700
+ "support": 24,
701
+ "precision_mean": 0.525708635464733,
702
+ "precision_sample_sd": 0.02235198659615819,
703
+ "recall_mean": 0.8888888888888888,
704
+ "recall_sample_sd": 0.048112522432468774,
705
+ "f1_mean": 0.6599065351235072,
706
+ "f1_sample_sd": 0.015362381542320084
707
+ },
708
+ {
709
+ "arm": "existing_specialized_separate_lineage",
710
+ "label": "Bandwagon,Reductio_ad_hitlerum",
711
+ "support": 7,
712
+ "precision_mean": 0.44485144485144484,
713
+ "precision_sample_sd": 0.016487284575115283,
714
+ "recall_mean": 0.7619047619047619,
715
+ "recall_sample_sd": 0.1649572197684645,
716
+ "f1_mean": 0.5571428571428572,
717
+ "f1_sample_sd": 0.051507875363771265
718
+ },
719
+ {
720
+ "arm": "existing_specialized_separate_lineage",
721
+ "label": "Black-and-White_Fallacy",
722
+ "support": 12,
723
+ "precision_mean": 0.4662644544997486,
724
+ "precision_sample_sd": 0.004538274146363688,
725
+ "recall_mean": 0.5833333333333334,
726
+ "recall_sample_sd": 0.08333333333333331,
727
+ "f1_mean": 0.516747552149851,
728
+ "f1_sample_sd": 0.03589484965697345
729
+ },
730
+ {
731
+ "arm": "existing_specialized_separate_lineage",
732
+ "label": "Causal_Oversimplification",
733
+ "support": 18,
734
+ "precision_mean": 0.4472934472934473,
735
+ "precision_sample_sd": 0.013055771210700413,
736
+ "recall_mean": 0.8333333333333333,
737
+ "recall_sample_sd": 0.1469861839480328,
738
+ "f1_mean": 0.5781794553724379,
739
+ "f1_sample_sd": 0.028407563015813122
740
+ },
741
+ {
742
+ "arm": "existing_specialized_separate_lineage",
743
+ "label": "Doubt",
744
+ "support": 22,
745
+ "precision_mean": 0.5045661001171619,
746
+ "precision_sample_sd": 0.023581174969843577,
747
+ "recall_mean": 1.0,
748
+ "recall_sample_sd": 0.0,
749
+ "f1_mean": 0.6704939668174963,
750
+ "f1_sample_sd": 0.020973132363659372
751
+ },
752
+ {
753
+ "arm": "existing_specialized_separate_lineage",
754
+ "label": "Exaggeration,Minimisation",
755
+ "support": 22,
756
+ "precision_mean": 0.44589793281653745,
757
+ "precision_sample_sd": 0.010987847993006481,
758
+ "recall_mean": 0.9393939393939394,
759
+ "recall_sample_sd": 0.06943296507508849,
760
+ "f1_mean": 0.6043956043956044,
761
+ "f1_sample_sd": 0.022305256187015846
762
+ },
763
+ {
764
+ "arm": "existing_specialized_separate_lineage",
765
+ "label": "Flag-Waving",
766
+ "support": 17,
767
+ "precision_mean": 0.5576388888888889,
768
+ "precision_sample_sd": 0.03705367103320583,
769
+ "recall_mean": 0.8823529411764706,
770
+ "recall_sample_sd": 0.11764705882352944,
771
+ "f1_mean": 0.680769868923179,
772
+ "f1_sample_sd": 0.04164658769406446
773
+ },
774
+ {
775
+ "arm": "existing_specialized_separate_lineage",
776
+ "label": "Loaded_Language",
777
+ "support": 45,
778
+ "precision_mean": 0.8387650085763294,
779
+ "precision_sample_sd": 0.01782556577257849,
780
+ "recall_mean": 1.0,
781
+ "recall_sample_sd": 0.0,
782
+ "f1_mean": 0.9122448979591837,
783
+ "f1_sample_sd": 0.01060439269940129
784
+ },
785
+ {
786
+ "arm": "existing_specialized_separate_lineage",
787
+ "label": "Name_Calling,Labeling",
788
+ "support": 33,
789
+ "precision_mean": 0.6957104190221673,
790
+ "precision_sample_sd": 0.019201490809269257,
791
+ "recall_mean": 0.9696969696969697,
792
+ "recall_sample_sd": 0.030303030303030276,
793
+ "f1_mean": 0.8101049442821594,
794
+ "f1_sample_sd": 0.022092733328802537
795
+ },
796
+ {
797
+ "arm": "existing_specialized_separate_lineage",
798
+ "label": "Repetition",
799
+ "support": 23,
800
+ "precision_mean": 0.4781557067271353,
801
+ "precision_sample_sd": 0.009897987322673213,
802
+ "recall_mean": 0.9420289855072463,
803
+ "recall_sample_sd": 0.06641414050660639,
804
+ "f1_mean": 0.6337774426009719,
805
+ "f1_sample_sd": 0.016444118333485254
806
+ },
807
+ {
808
+ "arm": "existing_specialized_separate_lineage",
809
+ "label": "Slogans",
810
+ "support": 14,
811
+ "precision_mean": 0.4334215167548501,
812
+ "precision_sample_sd": 0.05811581861282867,
813
+ "recall_mean": 0.6904761904761905,
814
+ "recall_sample_sd": 0.1649572197684645,
815
+ "f1_mean": 0.5201316298877274,
816
+ "f1_sample_sd": 0.018567913895656114
817
+ },
818
+ {
819
+ "arm": "existing_specialized_separate_lineage",
820
+ "label": "Thought-terminating_Cliches",
821
+ "support": 7,
822
+ "precision_mean": 0.18446115288220552,
823
+ "precision_sample_sd": 0.03641251252864963,
824
+ "recall_mean": 0.6666666666666666,
825
+ "recall_sample_sd": 0.1649572197684645,
826
+ "f1_mean": 0.2868626868626869,
827
+ "f1_sample_sd": 0.0511620864429468
828
+ },
829
+ {
830
+ "arm": "existing_specialized_separate_lineage",
831
+ "label": "Whataboutism,Straw_Men,Red_Herring",
832
+ "support": 10,
833
+ "precision_mean": 0.2777777777777778,
834
+ "precision_sample_sd": 0.0481125224324688,
835
+ "recall_mean": 0.4,
836
+ "recall_sample_sd": 0.0,
837
+ "f1_mean": 0.32634032634032634,
838
+ "f1_sample_sd": 0.03229931575886014
839
+ }
840
+ ],
841
+ "compression_ladder": [
842
+ {
843
+ "name": "24L upstream text teacher",
844
+ "layers": 24,
845
+ "parameters": 752393024,
846
+ "weight_bytes": 1504825632,
847
+ "weight_mib": 1435.1135559082031,
848
+ "weight_sha256": "30783da4651259c26b8a204821ea426b9526431e0c14da0d1c6fa284e7414e5e",
849
+ "kd_stage": "",
850
+ "mean_total_loss": "",
851
+ "mean_interface_loss": "",
852
+ "mean_final_loss": ""
853
+ },
854
+ {
855
+ "name": "8L task-free KD",
856
+ "layers": 8,
857
+ "parameters": 420318144,
858
+ "weight_bytes": 840647880,
859
+ "weight_mib": 801.7042922973633,
860
+ "weight_sha256": "086fbaf9a4838ecde56cd3541c9c3212b22d0c5ee745cf2c1c5e0bb31e0ec22c",
861
+ "kd_stage": "24to8",
862
+ "mean_total_loss": 0.17622653172702485,
863
+ "mean_interface_loss": 0.21562811763578793,
864
+ "mean_final_loss": 0.16626427527853593
865
+ },
866
+ {
867
+ "name": "6L task-free KD",
868
+ "layers": 6,
869
+ "parameters": 377207424,
870
+ "weight_bytes": 754423424,
871
+ "weight_mib": 719.4742431640625,
872
+ "weight_sha256": "492749c20f77a42f6bc993f0927a82217b94f872b7eacf25612f7520c9bc28ce",
873
+ "kd_stage": "8to6",
874
+ "mean_total_loss": 0.03382104352249371,
875
+ "mean_interface_loss": 0.04486069045515251,
876
+ "mean_final_loss": 0.031147060785087888
877
+ },
878
+ {
879
+ "name": "4L task-free KD root",
880
+ "layers": 4,
881
+ "parameters": 334096704,
882
+ "weight_bytes": 668198976,
883
+ "weight_mib": 637.2442016601562,
884
+ "weight_sha256": "2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3",
885
+ "kd_stage": "6to4",
886
+ "mean_total_loss": 0.047020394468859195,
887
+ "mean_interface_loss": 0.07048421185527332,
888
+ "mean_final_loss": 0.043119939065377366
889
+ }
890
+ ]
891
+ }
docs/RELEASE_CONTRACT.md CHANGED
@@ -14,14 +14,13 @@
14
  SemEval examples, labels, logits, thresholds, or evidence objectives.
15
  2. `models/semeval-propaganda/`: the existing three-seed task-specific
16
  classifier copied byte-for-byte as a separate historical lineage. It is not
17
- described as a fine-tuned child of the new root base.
18
  3. `distillation/`: the reusable 24L→8L→6L→4L platform for unlabeled
19
- representation distillation and user-owned single-label or multilabel
20
- classification data.
21
 
22
  ## Base training contract
23
 
24
- - Source: the text backbone and full tokenizer of `Qwen/Qwen3.5-0.8B`.
25
  - Structural defaults:
26
  - 24→8: `0,4,6,11,13,16,20,23`
27
  - 8→6: `0,1,3,4,6,7`
@@ -31,43 +30,81 @@
31
  - Objective: aligned hidden-boundary and final-representation distillation.
32
  - One epoch per stage, seed 41, maximum length 128, batch size 1, gradient
33
  accumulation 8, learning rate 2e-5, BF16 CUDA.
 
34
  - No classification head is published at the root.
35
 
36
- The WikiText dataset page currently has inconsistent license wording: metadata
37
- lists CC BY-SA 3.0 and GFDL, while the prose says CC BY-SA 4.0. The release
38
- redistributes no WikiText text and discloses both statements rather than
39
- silently selecting one.
40
 
41
- ## Transfer gate
42
 
43
- The only downstream validation task is an article-level multilabel task derived
44
- from SemEval-2020 Task 11. It is not the official span- or fragment-level task.
45
- The public test split contains 55 previously opened articles, so results are
46
- exploratory rather than confirmatory.
47
 
48
- | Initialization | Seed 41 | Seed 42 | Seed 43 | Mean ± sample SD |
49
- |---|---:|---:|---:|---:|
50
- | Task-agnostic KD base | 0.58831 | 0.60284 | 0.57128 | 0.58748 ± 0.01579 |
51
- | Structural 4L copy, no task-free KD | 0.51694 | 0.53494 | 0.50238 | 0.51809 ± 0.01631 |
 
 
 
52
 
53
- The preregistered publication gate was that the task-agnostic base must not be
54
- worse than the structural-copy control by more than 0.01 mean Macro-F1. The
55
- observed mean difference was +0.06939, so the gate passed. Three training seeds
56
- are not a confidence interval and no significance claim is made.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
 
58
  ## Release gates
59
 
60
  - Root `model.safetensors` contains no `score.weight`.
61
- - Root loads with `AutoModel`, and a temporary classification head can be
62
- attached with `AutoModelForSequenceClassification`.
63
  - Root weight SHA-256 is
64
  `2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3`.
65
  - Base manifests say `labels_read=false` and `semeval_used=false`.
66
- - Nested specialized checkpoint hashes equal the already published release.
 
 
67
  - The platform validates label order and ID/group split leakage, never
68
  overwrites output directories, hashes inputs/outputs, verifies immutable
69
  teachers, and reloads fresh checkpoints.
70
  - The release contains no SemEval or WikiText source data.
71
  - The old specialized repository revision and inventory are checked before and
72
- after creation of the new repository.
73
-
 
14
  SemEval examples, labels, logits, thresholds, or evidence objectives.
15
  2. `models/semeval-propaganda/`: the existing three-seed task-specific
16
  classifier copied byte-for-byte as a separate historical lineage. It is not
17
+ described as a fine-tuned child of the root base.
18
  3. `distillation/`: the reusable 24L→8L→6L→4L platform for unlabeled
19
+ representation distillation and user-owned single-label or multilabel data.
 
20
 
21
  ## Base training contract
22
 
23
+ - Source: text backbone and full tokenizer of `Qwen/Qwen3.5-0.8B`.
24
  - Structural defaults:
25
  - 24→8: `0,4,6,11,13,16,20,23`
26
  - 8→6: `0,1,3,4,6,7`
 
30
  - Objective: aligned hidden-boundary and final-representation distillation.
31
  - One epoch per stage, seed 41, maximum length 128, batch size 1, gradient
32
  accumulation 8, learning rate 2e-5, BF16 CUDA.
33
+ - Root artifact: 334,096,704 parameters and 668,198,976 BF16 weight bytes.
34
  - No classification head is published at the root.
35
 
36
+ WikiText source text is not redistributed. Its dataset page's metadata and
37
+ prose currently disagree on the license version, so users must review the
38
+ upstream terms directly.
 
39
 
40
+ ## Benchmark evidence contract
41
 
42
+ The headless root is evaluated through a fresh-head transfer probe, not by
43
+ pretending it is already a classifier. Both the task-agnostic base and the
44
+ structural-copy control use the same frozen protocol:
 
45
 
46
+ - SemEval-2020 Task 11 annotations converted to article-level 14-label targets;
47
+ - train/calibration/test article counts 260/56/55;
48
+ - seeds 41/42/43; five epochs; no early stopping;
49
+ - 256-token windows, stride 128, label-wise maximum article aggregation;
50
+ - micro batch 1, accumulation 32, AdamW, LR 2e-5, BF16;
51
+ - one global threshold selected on epoch-5 calibration Macro-F1 then Micro-F1;
52
+ - 55 previously opened test articles and 434 test windows.
53
 
54
+ | Initialization | Macro-F1 mean ± SD | Micro-F1 mean ± SD | Exact match mean ± SD |
55
+ |---|---:|---:|---:|
56
+ | Task-agnostic KD base | 0.58748 ± 0.01579 | 0.65295 ± 0.01088 | 0.03030 ± 0.02777 |
57
+ | Structural copy, no task-free KD | 0.51809 ± 0.01631 | 0.59046 ± 0.01603 | 0.00000 ± 0.00000 |
58
+
59
+ The observed same-seed mean Macro-F1 difference is +0.06939. All values and
60
+ sample SDs must be derived from the six frozen full reports at build time; the
61
+ card, JSON, and CSV tables must numerically close against those reports.
62
+
63
+ The existing specialized 4L lineage may appear only as a clearly labeled
64
+ reference because it uses a separate task-specific shrink/distillation history
65
+ and reduced vocabulary. It is not evidence that the task-free base produced
66
+ that checkpoint.
67
+
68
+ The report bundle must include:
69
+
70
+ - per-arm, per-seed Macro-F1, Micro-F1, exact match, positive rate, threshold;
71
+ - three-seed means and sample SDs;
72
+ - per-label precision, recall, F1, and fixed test support;
73
+ - parameters, CUDA allocation/reservation, total wall time, and test timing;
74
+ - compression-ladder depth, parameter count, BF16 size, hashes, and KD losses;
75
+ - normalized full frozen reports with article-level probability vectors.
76
+
77
+ The test is already opened, per-label supports are small, and three seeds are
78
+ not a confidence interval. No confirmatory, significance, universal
79
+ classification, or population-generalization claim is permitted.
80
+
81
+ ## Multilingual metadata contract
82
+
83
+ The root keeps the full upstream tokenizer. The official Qwen3.5 card states
84
+ support for 201 languages and dialects and reports upstream multilingual
85
+ benchmarks. Therefore `language: multilingual` and a `multilingual` tag are
86
+ permitted as inherited input/architecture metadata.
87
+
88
+ This release's task-free KD corpus and only downstream probe are English.
89
+ Accordingly, the card must state prominently that the four-layer root has no
90
+ direct multilingual downstream evaluation and that tokenizer coverage is not a
91
+ multilingual classification-quality result. It may not claim verified quality
92
+ across 201 languages.
93
 
94
  ## Release gates
95
 
96
  - Root `model.safetensors` contains no `score.weight`.
97
+ - Root loads with `AutoModel`; a temporary head can be attached with
98
+ `AutoModelForSequenceClassification`.
99
  - Root weight SHA-256 is
100
  `2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3`.
101
  - Base manifests say `labels_read=false` and `semeval_used=false`.
102
+ - Root metadata contains `multilingual` and contains no propaganda task tag.
103
+ - Benchmark CSV/JSON/card values close against all frozen reports.
104
+ - Nested specialized checkpoint hashes equal the existing release.
105
  - The platform validates label order and ID/group split leakage, never
106
  overwrites output directories, hashes inputs/outputs, verifies immutable
107
  teachers, and reloads fresh checkpoints.
108
  - The release contains no SemEval or WikiText source data.
109
  - The old specialized repository revision and inventory are checked before and
110
+ after updating only the new repository.
 
models/semeval-propaganda/README.md CHANGED
@@ -23,5 +23,7 @@ model = AutoModelForSequenceClassification.from_pretrained(
23
  ```
24
 
25
  The files below are byte-identical copies of the current release bundle. The
26
- original release manifest and locked thresholds are included for provenance.
27
- No SemEval source article or annotation file is redistributed.
 
 
 
23
  ```
24
 
25
  The files below are byte-identical copies of the current release bundle. The
26
+ original release manifest, locked thresholds, three full reports, and aggregate
27
+ are included for provenance. The repository-level `benchmark/` directory also
28
+ places this lineage beside the task-agnostic transfer probe as a clearly marked
29
+ reference. No SemEval source article or annotation file is redistributed.
models/semeval-propaganda/seeds/seed42/tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff
 
models/semeval-propaganda/seeds/seed43/tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff
 
models/semeval-propaganda/tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff
 
release_manifest.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "schema": "qwen35-unified-classification-base-release-v1",
3
  "status": "complete",
4
  "repo_id": "mp-juuuns/qwen35-standalone4l-classification-base",
5
  "old_repo_preserved": {
@@ -8,6 +8,8 @@
8
  },
9
  "root_role": "task_agnostic_headless_classification_backbone",
10
  "root_weight_sha256": "2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3",
 
 
11
  "nested_specialized_is_separate_lineage": true,
12
  "source_data_redistributed": false,
13
  "files_before_manifest": [
@@ -28,57 +30,134 @@
28
  {
29
  "path": "README.md",
30
  "role": "root model card",
31
- "bytes": 5628,
32
- "sha256": "4f5af67b76f19fca88e242298fa501aa6aa811b4f1f51b481d17abb46e0d39cf",
33
  "source": "docs/huggingface/UNIFIED_BASE_RELEASE_MODEL_CARD.md"
34
  },
35
  {
36
- "path": "benchmark/reports/structural_control-seed41.json",
37
- "role": "normalized transfer report",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
  "bytes": 115612,
39
  "sha256": "816c3249d3ec281ef73d34dc3ab7576a48415d250c32977eb94ca37f73b0b810",
40
  "source": "generated"
41
  },
42
  {
43
- "path": "benchmark/reports/structural_control-seed42.json",
44
- "role": "normalized transfer report",
45
  "bytes": 115225,
46
  "sha256": "0b19eabe1e31d0012c754db3b31173f8a6fcdee30973f1a1bebd3a370d11064c",
47
  "source": "generated"
48
  },
49
  {
50
- "path": "benchmark/reports/structural_control-seed43.json",
51
- "role": "normalized transfer report",
52
  "bytes": 115744,
53
  "sha256": "6dde34173462eac50141ff082c18194f3233f8c06853532d9b12756562e69e2e",
54
  "source": "generated"
55
  },
56
  {
57
- "path": "benchmark/reports/task_agnostic-seed41.json",
58
- "role": "normalized transfer report",
59
  "bytes": 115885,
60
  "sha256": "6b3a2e0372a83e1e71c2eca9596efc8cd1a3e39c28e25e3373f195431b5858a4",
61
  "source": "generated"
62
  },
63
  {
64
- "path": "benchmark/reports/task_agnostic-seed42.json",
65
- "role": "normalized transfer report",
66
  "bytes": 115674,
67
  "sha256": "42dc788983716be0e9847aad87ff79c7e7297884f2635a8d098639be77326639",
68
  "source": "generated"
69
  },
70
  {
71
- "path": "benchmark/reports/task_agnostic-seed43.json",
72
- "role": "normalized transfer report",
73
  "bytes": 115062,
74
  "sha256": "7ee52dc94a152a2f767dcff171f3c9cc1072fabd3ebc7b14db53617b7b1d33d5",
75
  "source": "generated"
76
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
  {
78
  "path": "benchmark/semeval-transfer-summary.json",
79
- "role": "machine-readable one-task transfer summary",
80
- "bytes": 1190,
81
- "sha256": "18c8d840dd2290ca25c517d52cbcf1ad6c6ae8d9c32c545236e96bd5ada7dab2",
 
 
 
 
 
 
 
82
  "source": "generated"
83
  },
84
  {
@@ -203,8 +282,8 @@
203
  {
204
  "path": "docs/RELEASE_CONTRACT.md",
205
  "role": "release contract",
206
- "bytes": 3399,
207
- "sha256": "5a1d9e939cd0f350770bcc2a116bc427bb9971bc4fbbbda17ce568e9127185ef",
208
  "source": "docs/huggingface/UNIFIED_BASE_RELEASE_CONTRACT.md"
209
  },
210
  {
@@ -231,8 +310,8 @@
231
  {
232
  "path": "models/semeval-propaganda/README.md",
233
  "role": "specialized lineage card",
234
- "bytes": 1231,
235
- "sha256": "8dbc2e641280c471b78064808ced7fffcb81c639784a5936b57175975a718f6f",
236
  "source": "generated"
237
  },
238
  {
 
1
  {
2
+ "schema": "qwen35-unified-classification-base-release-v2",
3
  "status": "complete",
4
  "repo_id": "mp-juuuns/qwen35-standalone4l-classification-base",
5
  "old_repo_preserved": {
 
8
  },
9
  "root_role": "task_agnostic_headless_classification_backbone",
10
  "root_weight_sha256": "2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3",
11
+ "root_multilingual_metadata_scope": "inherited tokenizer/architecture coverage; no direct multilingual downstream evaluation",
12
+ "benchmark_schema": "standalone4l-complete-transfer-benchmark-v2",
13
  "nested_specialized_is_separate_lineage": true,
14
  "source_data_redistributed": false,
15
  "files_before_manifest": [
 
30
  {
31
  "path": "README.md",
32
  "role": "root model card",
33
+ "bytes": 12909,
34
+ "sha256": "5fcf007de6b6e828711c874310de1e042f1dada6989c70930ac0881285a9dc61",
35
  "source": "docs/huggingface/UNIFIED_BASE_RELEASE_MODEL_CARD.md"
36
  },
37
  {
38
+ "path": "benchmark/BENCHMARK_CARD.md",
39
+ "role": "generated benchmark report",
40
+ "bytes": 3173,
41
+ "sha256": "bf4073de38c3905effd2dcce6f38894f325c8a0a79bedfb2779f0545c86b9c9f",
42
+ "source": "generated"
43
+ },
44
+ {
45
+ "path": "benchmark/arm_summary.csv",
46
+ "role": "three-seed metric summary",
47
+ "bytes": 1655,
48
+ "sha256": "e01bd7ac59a3212e44b77e5fcfbc7c41ec91d74c998306d2438209d6ef1a3aa7",
49
+ "source": "generated"
50
+ },
51
+ {
52
+ "path": "benchmark/compression_ladder.csv",
53
+ "role": "24L to 4L compression evidence",
54
+ "bytes": 827,
55
+ "sha256": "2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b",
56
+ "source": "generated"
57
+ },
58
+ {
59
+ "path": "benchmark/per_label_metrics.csv",
60
+ "role": "per-label metrics by arm and seed",
61
+ "bytes": 12963,
62
+ "sha256": "ffc21be32b2106c7bf1df3854097a975ac370da01ac1b732eb5c2ba9075ddeda",
63
+ "source": "generated"
64
+ },
65
+ {
66
+ "path": "benchmark/per_label_summary.csv",
67
+ "role": "per-label three-seed summary",
68
+ "bytes": 7009,
69
+ "sha256": "c8e3876abf6cfe960c9332d3bc9d1fbde2f3213f3111351acf83b21bbb184049",
70
+ "source": "generated"
71
+ },
72
+ {
73
+ "path": "benchmark/reports/existing_specialized_separate_lineage-seed41.json",
74
+ "role": "normalized frozen benchmark report",
75
+ "bytes": 115793,
76
+ "sha256": "c1bd379680abcf50b47b7d23e4aafbc8b5e7f40a0b0e9700bde21a8ce49af803",
77
+ "source": "generated"
78
+ },
79
+ {
80
+ "path": "benchmark/reports/existing_specialized_separate_lineage-seed42.json",
81
+ "role": "normalized frozen benchmark report",
82
+ "bytes": 115780,
83
+ "sha256": "a30e38433d008db4aa7e5fbfd477ab80a0daf50e970f8e7b04397efc1a47843a",
84
+ "source": "generated"
85
+ },
86
+ {
87
+ "path": "benchmark/reports/existing_specialized_separate_lineage-seed43.json",
88
+ "role": "normalized frozen benchmark report",
89
+ "bytes": 115985,
90
+ "sha256": "96466290d09d27b8bc044a9f9a95765ce12b4d7b02dfb0123e436e7535623353",
91
+ "source": "generated"
92
+ },
93
+ {
94
+ "path": "benchmark/reports/structural_copy_control-seed41.json",
95
+ "role": "normalized frozen benchmark report",
96
  "bytes": 115612,
97
  "sha256": "816c3249d3ec281ef73d34dc3ab7576a48415d250c32977eb94ca37f73b0b810",
98
  "source": "generated"
99
  },
100
  {
101
+ "path": "benchmark/reports/structural_copy_control-seed42.json",
102
+ "role": "normalized frozen benchmark report",
103
  "bytes": 115225,
104
  "sha256": "0b19eabe1e31d0012c754db3b31173f8a6fcdee30973f1a1bebd3a370d11064c",
105
  "source": "generated"
106
  },
107
  {
108
+ "path": "benchmark/reports/structural_copy_control-seed43.json",
109
+ "role": "normalized frozen benchmark report",
110
  "bytes": 115744,
111
  "sha256": "6dde34173462eac50141ff082c18194f3233f8c06853532d9b12756562e69e2e",
112
  "source": "generated"
113
  },
114
  {
115
+ "path": "benchmark/reports/task_agnostic_base-seed41.json",
116
+ "role": "normalized frozen benchmark report",
117
  "bytes": 115885,
118
  "sha256": "6b3a2e0372a83e1e71c2eca9596efc8cd1a3e39c28e25e3373f195431b5858a4",
119
  "source": "generated"
120
  },
121
  {
122
+ "path": "benchmark/reports/task_agnostic_base-seed42.json",
123
+ "role": "normalized frozen benchmark report",
124
  "bytes": 115674,
125
  "sha256": "42dc788983716be0e9847aad87ff79c7e7297884f2635a8d098639be77326639",
126
  "source": "generated"
127
  },
128
  {
129
+ "path": "benchmark/reports/task_agnostic_base-seed43.json",
130
+ "role": "normalized frozen benchmark report",
131
  "bytes": 115062,
132
  "sha256": "7ee52dc94a152a2f767dcff171f3c9cc1072fabd3ebc7b14db53617b7b1d33d5",
133
  "source": "generated"
134
  },
135
+ {
136
+ "path": "benchmark/resource_metrics.csv",
137
+ "role": "resource measurements by arm and seed",
138
+ "bytes": 1649,
139
+ "sha256": "bf5cefdec3b2f3646114bd18bbf8e5e39721bde768ba968d73f22e96fc9ca8cf",
140
+ "source": "generated"
141
+ },
142
+ {
143
+ "path": "benchmark/seed_metrics.csv",
144
+ "role": "overall metrics by arm and seed",
145
+ "bytes": 1175,
146
+ "sha256": "e0b08a958d25243158a03dff76ca153d2ded76cc3f1771e4046a0068688ad6a8",
147
+ "source": "generated"
148
+ },
149
  {
150
  "path": "benchmark/semeval-transfer-summary.json",
151
+ "role": "compatibility alias for complete transfer summary",
152
+ "bytes": 26913,
153
+ "sha256": "3d2c9fcf9c6cc7da1a885968b70a42eaa38659d317c9d3290a2b3066f3f9ee6a",
154
+ "source": "generated"
155
+ },
156
+ {
157
+ "path": "benchmark/summary.json",
158
+ "role": "complete machine-readable transfer benchmark",
159
+ "bytes": 26913,
160
+ "sha256": "3d2c9fcf9c6cc7da1a885968b70a42eaa38659d317c9d3290a2b3066f3f9ee6a",
161
  "source": "generated"
162
  },
163
  {
 
282
  {
283
  "path": "docs/RELEASE_CONTRACT.md",
284
  "role": "release contract",
285
+ "bytes": 5381,
286
+ "sha256": "1b6e475b9d7d9af52bb8845de020303b1c6e89646d8e4f9be17dbe4ab2910417",
287
  "source": "docs/huggingface/UNIFIED_BASE_RELEASE_CONTRACT.md"
288
  },
289
  {
 
310
  {
311
  "path": "models/semeval-propaganda/README.md",
312
  "role": "specialized lineage card",
313
+ "bytes": 1403,
314
+ "sha256": "bf4ba7cc2514502a28f424507c41c1cfce5089c051ba7500cf46c1d13860722f",
315
  "source": "generated"
316
  },
317
  {