Feature Extraction
Transformers
Safetensors
multilingual
qwen3_5_text
qwen3.5
classification-backbone
text-classification
knowledge-distillation
model-compression
edge-ai
Instructions to use mp-juuuns/qwen35-standalone4l-classification-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mp-juuuns/qwen35-standalone4l-classification-base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="mp-juuuns/qwen35-standalone4l-classification-base")# Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("mp-juuuns/qwen35-standalone4l-classification-base") model = AutoModel.from_pretrained("mp-juuuns/qwen35-standalone4l-classification-base", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add Matplotlib benchmark visualizations and source data
Browse files- .gitattributes +2 -3
- README.md +25 -0
- SHA256SUMS +17 -2
- benchmark/figures/01_overall_transfer_quality.csv +7 -0
- benchmark/figures/01_overall_transfer_quality.pdf +0 -0
- benchmark/figures/01_overall_transfer_quality.png +0 -0
- benchmark/figures/02_per_label_transfer_metrics.csv +15 -0
- benchmark/figures/02_per_label_transfer_metrics.pdf +0 -0
- benchmark/figures/02_per_label_transfer_metrics.png +3 -0
- benchmark/figures/03_quality_efficiency.csv +4 -0
- benchmark/figures/03_quality_efficiency.pdf +0 -0
- benchmark/figures/03_quality_efficiency.png +0 -0
- benchmark/figures/04_compression_ladder.csv +5 -0
- benchmark/figures/04_compression_ladder.pdf +0 -0
- benchmark/figures/04_compression_ladder.png +3 -0
- benchmark/figures/README.md +13 -0
- benchmark/figures/manifest.json +84 -0
- benchmark/figures/source_summary.json +891 -0
- release_manifest.json +109 -3
.gitattributes
CHANGED
|
@@ -1,4 +1,3 @@
|
|
| 1 |
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
models/semeval-propaganda/tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 1 |
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
benchmark/figures/02_per_label_transfer_metrics.png filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
benchmark/figures/04_compression_ladder.png filter=lfs diff=lfs merge=lfs -text
|
|
|
README.md
CHANGED
|
@@ -101,6 +101,12 @@ parameter counts, BF16 file sizes, hashes, and stage values are in
|
|
| 101 |
[`benchmark/compression_ladder.csv`](benchmark/compression_ladder.csv) and
|
| 102 |
[`provenance/task_agnostic_stages/`](provenance/task_agnostic_stages/).
|
| 103 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
## Downstream benchmark contract
|
| 105 |
|
| 106 |
Because the root is headless, it cannot be scored as a classifier without
|
|
@@ -158,6 +164,12 @@ The task-agnostic KD initialization improved same-seed mean Macro-F1 by
|
|
| 158 |
`+0.06939` over the structural-copy control. This is an observed comparison
|
| 159 |
under this protocol, not a significance or universal-superiority claim.
|
| 160 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 161 |
### Task-agnostic base: per-label test results
|
| 162 |
|
| 163 |
Precision, recall, and F1 are three-seed means. Support is the fixed number of
|
|
@@ -184,6 +196,11 @@ Per-seed values and sample SDs for all labels and all arms are available in
|
|
| 184 |
[`benchmark/per_label_metrics.csv`](benchmark/per_label_metrics.csv) and
|
| 185 |
[`benchmark/per_label_summary.csv`](benchmark/per_label_summary.csv).
|
| 186 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 187 |
### Efficiency and resource measurements
|
| 188 |
|
| 189 |
These numbers belong to the fresh-head transfer checkpoints, not a standalone
|
|
@@ -203,6 +220,12 @@ about CPU, mobile, UNO Q, or other deployment performance. Full seed-level
|
|
| 203 |
resource fields are in
|
| 204 |
[`benchmark/resource_metrics.csv`](benchmark/resource_metrics.csv).
|
| 205 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 206 |
### Machine-readable benchmark bundle
|
| 207 |
|
| 208 |
- [`benchmark/BENCHMARK_CARD.md`](benchmark/BENCHMARK_CARD.md): generated
|
|
@@ -220,6 +243,8 @@ resource fields are in
|
|
| 220 |
parameters, CUDA memory, wall time, and test timing
|
| 221 |
- [`benchmark/reports/`](benchmark/reports/): normalized full frozen reports,
|
| 222 |
including article-level probability vectors
|
|
|
|
|
|
|
| 223 |
|
| 224 |
## Multilingual scope
|
| 225 |
|
|
|
|
| 101 |
[`benchmark/compression_ladder.csv`](benchmark/compression_ladder.csv) and
|
| 102 |
[`provenance/task_agnostic_stages/`](provenance/task_agnostic_stages/).
|
| 103 |
|
| 104 |
+

|
| 105 |
+
|
| 106 |
+
The chart reports the actual unsharded safetensors parameter counts and BF16
|
| 107 |
+
file sizes. It does not imply that downstream quality scales monotonically with
|
| 108 |
+
layer count.
|
| 109 |
+
|
| 110 |
## Downstream benchmark contract
|
| 111 |
|
| 112 |
Because the root is headless, it cannot be scored as a classifier without
|
|
|
|
| 164 |
`+0.06939` over the structural-copy control. This is an observed comparison
|
| 165 |
under this protocol, not a significance or universal-superiority claim.
|
| 166 |
|
| 167 |
+

|
| 168 |
+
|
| 169 |
+
Bars are three-seed means and whiskers are sample SD. The specialized 4L point
|
| 170 |
+
is a separate historical lineage rather than a fine-tuned child of the root
|
| 171 |
+
base.
|
| 172 |
+
|
| 173 |
### Task-agnostic base: per-label test results
|
| 174 |
|
| 175 |
Precision, recall, and F1 are three-seed means. Support is the fixed number of
|
|
|
|
| 196 |
[`benchmark/per_label_metrics.csv`](benchmark/per_label_metrics.csv) and
|
| 197 |
[`benchmark/per_label_summary.csv`](benchmark/per_label_summary.csv).
|
| 198 |
|
| 199 |
+

|
| 200 |
+
|
| 201 |
+
Labels are ordered by their three-seed mean F1. Support remains visible because
|
| 202 |
+
several classes have only 7–14 positive test articles.
|
| 203 |
+
|
| 204 |
### Efficiency and resource measurements
|
| 205 |
|
| 206 |
These numbers belong to the fresh-head transfer checkpoints, not a standalone
|
|
|
|
| 220 |
resource fields are in
|
| 221 |
[`benchmark/resource_metrics.csv`](benchmark/resource_metrics.csv).
|
| 222 |
|
| 223 |
+

|
| 224 |
+
|
| 225 |
+
These plots compare the recorded fresh-head transfer checkpoints only.
|
| 226 |
+
Apparent differences must not be generalized to the headless root on untested
|
| 227 |
+
hardware.
|
| 228 |
+
|
| 229 |
### Machine-readable benchmark bundle
|
| 230 |
|
| 231 |
- [`benchmark/BENCHMARK_CARD.md`](benchmark/BENCHMARK_CARD.md): generated
|
|
|
|
| 243 |
parameters, CUDA memory, wall time, and test timing
|
| 244 |
- [`benchmark/reports/`](benchmark/reports/): normalized full frozen reports,
|
| 245 |
including article-level probability vectors
|
| 246 |
+
- [`benchmark/figures/`](benchmark/figures/): Matplotlib PNG/PDF figures,
|
| 247 |
+
per-figure source CSVs, and a hash manifest
|
| 248 |
|
| 249 |
## Multilingual scope
|
| 250 |
|
SHA256SUMS
CHANGED
|
@@ -1,9 +1,24 @@
|
|
| 1 |
ee05350c7fda0af4da116020dd7b976d731de3beed4f5e717d5bcc95ead24550 .gitattributes
|
| 2 |
77fd4710def9ec3c0f6225800e0235f15a425abd4a8b03559127fcd782612049 LICENSE
|
| 3 |
-
|
| 4 |
bf4073de38c3905effd2dcce6f38894f325c8a0a79bedfb2779f0545c86b9c9f benchmark/BENCHMARK_CARD.md
|
| 5 |
e01bd7ac59a3212e44b77e5fcfbc7c41ec91d74c998306d2438209d6ef1a3aa7 benchmark/arm_summary.csv
|
| 6 |
2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b benchmark/compression_ladder.csv
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
ffc21be32b2106c7bf1df3854097a975ac370da01ac1b732eb5c2ba9075ddeda benchmark/per_label_metrics.csv
|
| 8 |
c8e3876abf6cfe960c9332d3bc9d1fbde2f3213f3111351acf83b21bbb184049 benchmark/per_label_summary.csv
|
| 9 |
c1bd379680abcf50b47b7d23e4aafbc8b5e7f40a0b0e9700bde21a8ce49af803 benchmark/reports/existing_specialized_separate_lineage-seed41.json
|
|
@@ -70,7 +85,7 @@ ec734855870758cbfef99031dac2ded85b54c496532e5c514a9a4f63287c3493 provenance/bas
|
|
| 70 |
b9ce99b993c4f4563bc2521fc731bfd82f5e8149c141133599fbd8f982ba43d9 provenance/task_agnostic_stages/24to8.json
|
| 71 |
ec734855870758cbfef99031dac2ded85b54c496532e5c514a9a4f63287c3493 provenance/task_agnostic_stages/6to4.json
|
| 72 |
65e5383afbbcfe0b7dd78a5411c319ed6bf2586659fb1892aca9ecbd5dfd4993 provenance/task_agnostic_stages/8to6.json
|
| 73 |
-
|
| 74 |
7b48e7811e56e552fc4aef44189e1d7c9b1352c85d2b3f10ff0bdd1ea5cdd824 requirements.txt
|
| 75 |
06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523 tokenizer.json
|
| 76 |
5ab9bed0a4d27949672f65ba1141d6dd5b0514fb9091d3426506ecebb5d5e294 tokenizer_config.json
|
|
|
|
| 1 |
ee05350c7fda0af4da116020dd7b976d731de3beed4f5e717d5bcc95ead24550 .gitattributes
|
| 2 |
77fd4710def9ec3c0f6225800e0235f15a425abd4a8b03559127fcd782612049 LICENSE
|
| 3 |
+
49cb7082e65f26835c90131ce7ff05f47ebd77ec9352dad491a0083ede70bd50 README.md
|
| 4 |
bf4073de38c3905effd2dcce6f38894f325c8a0a79bedfb2779f0545c86b9c9f benchmark/BENCHMARK_CARD.md
|
| 5 |
e01bd7ac59a3212e44b77e5fcfbc7c41ec91d74c998306d2438209d6ef1a3aa7 benchmark/arm_summary.csv
|
| 6 |
2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b benchmark/compression_ladder.csv
|
| 7 |
+
769a516e53417e16b61cf30d95a02f6c812c0b1536242cdbf836da779753efbd benchmark/figures/01_overall_transfer_quality.csv
|
| 8 |
+
7221e0d2feb356e4d7f44aa0381e576254f734837e3f54fb0dfb98795ec19ded benchmark/figures/01_overall_transfer_quality.pdf
|
| 9 |
+
6d35b5964791cfff2701e43831996a9aad7d607305ab3031b0782bed2ffffa66 benchmark/figures/01_overall_transfer_quality.png
|
| 10 |
+
50a3ceb0e3146ba8cade9c86c4b59c7db6cbedfc41fac39a811d7f39f4bdeb0d benchmark/figures/02_per_label_transfer_metrics.csv
|
| 11 |
+
10756816e8ad800f5d872907986680abf637e76fe47f9790f27645f8f231ae4a benchmark/figures/02_per_label_transfer_metrics.pdf
|
| 12 |
+
5b558738b405a7b04bb30f38b6e53af21b2392ea12cad0d777e9d6d6985d3d04 benchmark/figures/02_per_label_transfer_metrics.png
|
| 13 |
+
f2a34cbcd700b2663cf761a5fb1dbd3aa99f310a96d4b4477a3a4529bee1a1b3 benchmark/figures/03_quality_efficiency.csv
|
| 14 |
+
18c81ce50fc7871b000c9e387f0f7d7780002cb2bd249153aadfa55a186342c0 benchmark/figures/03_quality_efficiency.pdf
|
| 15 |
+
08ed7ca8193d5f9065645fb22491ab88ee260d43566e9013aa0f98d7ab4dde87 benchmark/figures/03_quality_efficiency.png
|
| 16 |
+
2dfac2b8330724ce46a22d8f3ddfeffc767f52b93a93cb1a833c89c5820ffccc benchmark/figures/04_compression_ladder.csv
|
| 17 |
+
953d6e356498127b2a607530fac2e41b07b4bf38ed0af945f778b784dc6f7b23 benchmark/figures/04_compression_ladder.pdf
|
| 18 |
+
9f6c1d6fdefd45711fcfa57a7d0d91f6baaa3a5ee0b44a36554a9c74febe7090 benchmark/figures/04_compression_ladder.png
|
| 19 |
+
7e00107e6576da117689dca0c164c497ce32e9d93c034331e2e161625afc2092 benchmark/figures/README.md
|
| 20 |
+
b3e3da2fcf1ae8ea9c88019c289e3d45e7945df655218d5c0d3b905769811f50 benchmark/figures/manifest.json
|
| 21 |
+
3d2c9fcf9c6cc7da1a885968b70a42eaa38659d317c9d3290a2b3066f3f9ee6a benchmark/figures/source_summary.json
|
| 22 |
ffc21be32b2106c7bf1df3854097a975ac370da01ac1b732eb5c2ba9075ddeda benchmark/per_label_metrics.csv
|
| 23 |
c8e3876abf6cfe960c9332d3bc9d1fbde2f3213f3111351acf83b21bbb184049 benchmark/per_label_summary.csv
|
| 24 |
c1bd379680abcf50b47b7d23e4aafbc8b5e7f40a0b0e9700bde21a8ce49af803 benchmark/reports/existing_specialized_separate_lineage-seed41.json
|
|
|
|
| 85 |
b9ce99b993c4f4563bc2521fc731bfd82f5e8149c141133599fbd8f982ba43d9 provenance/task_agnostic_stages/24to8.json
|
| 86 |
ec734855870758cbfef99031dac2ded85b54c496532e5c514a9a4f63287c3493 provenance/task_agnostic_stages/6to4.json
|
| 87 |
65e5383afbbcfe0b7dd78a5411c319ed6bf2586659fb1892aca9ecbd5dfd4993 provenance/task_agnostic_stages/8to6.json
|
| 88 |
+
a37047ec47182dba0a5d9c455f8d3f1d4ec852b67e3229301802058f973e2ebf release_manifest.json
|
| 89 |
7b48e7811e56e552fc4aef44189e1d7c9b1352c85d2b3f10ff0bdd1ea5cdd824 requirements.txt
|
| 90 |
06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523 tokenizer.json
|
| 91 |
5ab9bed0a4d27949672f65ba1141d6dd5b0514fb9091d3426506ecebb5d5e294 tokenizer_config.json
|
benchmark/figures/01_overall_transfer_quality.csv
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
arm,metric,mean,sample_sd,n_seeds
|
| 2 |
+
task_agnostic_base,macro_f1,0.5874762842563245,0.01579462678670608,3
|
| 3 |
+
task_agnostic_base,micro_f1,0.6529504772586388,0.010884055708931178,3
|
| 4 |
+
structural_copy_control,macro_f1,0.5180869574433028,0.016311512588628545,3
|
| 5 |
+
structural_copy_control,micro_f1,0.5904584952122559,0.01602830914762526,3
|
| 6 |
+
existing_specialized_separate_lineage,macro_f1,0.5875679559555907,0.007165753447433348,3
|
| 7 |
+
existing_specialized_separate_lineage,micro_f1,0.647978512363484,0.0018739735696546695,3
|
benchmark/figures/01_overall_transfer_quality.pdf
ADDED
|
Binary file (18.1 kB). View file
|
|
|
benchmark/figures/01_overall_transfer_quality.png
ADDED
|
benchmark/figures/02_per_label_transfer_metrics.csv
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
label,support,precision_mean,recall_mean,f1_mean
|
| 2 |
+
Thought-terminating_Cliches,7,0.3492063492063492,0.3333333333333333,0.319014319014319
|
| 3 |
+
"Whataboutism,Straw_Men,Red_Herring",10,0.2761904761904762,0.5,0.330448343079922
|
| 4 |
+
Black-and-White_Fallacy,12,0.3667929292929293,0.47222222222222227,0.4053892966936445
|
| 5 |
+
Appeal_to_Authority,11,0.3958614484930274,0.5757575757575757,0.4657088122605364
|
| 6 |
+
"Exaggeration,Minimisation",22,0.4620811287477954,0.7727272727272727,0.5727040816326531
|
| 7 |
+
Slogans,14,0.5161361626878869,0.7380952380952381,0.5834824090638044
|
| 8 |
+
"Bandwagon,Reductio_ad_hitlerum",7,0.4851851851851852,0.8095238095238095,0.6006778309409889
|
| 9 |
+
Repetition,23,0.5196620583717357,0.7536231884057971,0.6147520781519571
|
| 10 |
+
Appeal_to_fear-prejudice,24,0.5540244550770866,0.7361111111111112,0.6277521761392729
|
| 11 |
+
Causal_Oversimplification,18,0.5428824049513704,0.7962962962962963,0.6446992723588468
|
| 12 |
+
Flag-Waving,17,0.5642642642642642,0.8235294117647058,0.6464506172839506
|
| 13 |
+
Doubt,22,0.5223832528180354,0.9848484848484849,0.6802057822343391
|
| 14 |
+
"Name_Calling,Labeling",33,0.7380952380952381,0.9090909090909091,0.8125087983911513
|
| 15 |
+
Loaded_Language,45,0.8649032907793016,0.9851851851851852,0.9208741623431562
|
benchmark/figures/02_per_label_transfer_metrics.pdf
ADDED
|
Binary file (21.4 kB). View file
|
|
|
benchmark/figures/02_per_label_transfer_metrics.png
ADDED
|
Git LFS Details
|
benchmark/figures/03_quality_efficiency.csv
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
arm,macro_f1_mean,macro_f1_sample_sd,test_ms_per_article_mean,peak_cuda_allocated_gib_mean,parameters_mean
|
| 2 |
+
task_agnostic_base,0.5874762842563245,0.01579462678670608,81.30633847886752,3.1487789154052734,334111040
|
| 3 |
+
structural_copy_control,0.5180869574433028,0.016311512588628545,81.16211107893226,3.1487789154052734,334111040
|
| 4 |
+
existing_specialized_separate_lineage,0.5875679559555907,0.007165753447433348,85.43555729062946,1.9964351654052734,210903360
|
benchmark/figures/03_quality_efficiency.pdf
ADDED
|
Binary file (21.9 kB). View file
|
|
|
benchmark/figures/03_quality_efficiency.png
ADDED
|
benchmark/figures/04_compression_ladder.csv
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
name,layers,parameters_millions,weight_mib,kd_stage,mean_total_loss
|
| 2 |
+
24L upstream text teacher,24,752.393024,1435.1135559082031,,
|
| 3 |
+
8L task-free KD,8,420.318144,801.7042922973633,24to8,0.17622653172702485
|
| 4 |
+
6L task-free KD,6,377.207424,719.4742431640625,8to6,0.03382104352249371
|
| 5 |
+
4L task-free KD root,4,334.096704,637.2442016601562,6to4,0.047020394468859195
|
benchmark/figures/04_compression_ladder.pdf
ADDED
|
Binary file (19.5 kB). View file
|
|
|
benchmark/figures/04_compression_ladder.png
ADDED
|
Git LFS Details
|
benchmark/figures/README.md
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Matplotlib benchmark figures
|
| 2 |
+
|
| 3 |
+
All figures are deterministically rendered from the frozen v2 benchmark bundle.
|
| 4 |
+
PNG files are intended for the Hugging Face model card; PDF files are vector
|
| 5 |
+
exports for reports. Each plot has a compact source CSV beside it.
|
| 6 |
+
|
| 7 |
+
1. Overall Macro/Micro-F1: three-seed means with sample-SD whiskers.
|
| 8 |
+
2. Per-label precision/recall/F1 for the task-agnostic base.
|
| 9 |
+
3. Macro-F1 against article inference time and peak CUDA allocation.
|
| 10 |
+
4. Parameters and BF16 size across the 24L→8L→6L→4L task-free ladder.
|
| 11 |
+
|
| 12 |
+
The 55-article test split was previously opened. Error bars are not confidence
|
| 13 |
+
intervals. The specialized checkpoint is a separate reference lineage.
|
benchmark/figures/manifest.json
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "standalone4l-matplotlib-figures-v1",
|
| 3 |
+
"status": "complete",
|
| 4 |
+
"source_benchmark_schema": "standalone4l-complete-transfer-benchmark-v2",
|
| 5 |
+
"source_summary_sha256": "3d2c9fcf9c6cc7da1a885968b70a42eaa38659d317c9d3290a2b3066f3f9ee6a",
|
| 6 |
+
"figures": [
|
| 7 |
+
"01_overall_transfer_quality",
|
| 8 |
+
"02_per_label_transfer_metrics",
|
| 9 |
+
"03_quality_efficiency",
|
| 10 |
+
"04_compression_ladder"
|
| 11 |
+
],
|
| 12 |
+
"files_before_manifest": [
|
| 13 |
+
{
|
| 14 |
+
"path": "01_overall_transfer_quality.csv",
|
| 15 |
+
"bytes": 500,
|
| 16 |
+
"sha256": "769a516e53417e16b61cf30d95a02f6c812c0b1536242cdbf836da779753efbd"
|
| 17 |
+
},
|
| 18 |
+
{
|
| 19 |
+
"path": "01_overall_transfer_quality.pdf",
|
| 20 |
+
"bytes": 18071,
|
| 21 |
+
"sha256": "7221e0d2feb356e4d7f44aa0381e576254f734837e3f54fb0dfb98795ec19ded"
|
| 22 |
+
},
|
| 23 |
+
{
|
| 24 |
+
"path": "01_overall_transfer_quality.png",
|
| 25 |
+
"bytes": 82127,
|
| 26 |
+
"sha256": "6d35b5964791cfff2701e43831996a9aad7d607305ab3031b0782bed2ffffa66"
|
| 27 |
+
},
|
| 28 |
+
{
|
| 29 |
+
"path": "02_per_label_transfer_metrics.csv",
|
| 30 |
+
"bytes": 1169,
|
| 31 |
+
"sha256": "50a3ceb0e3146ba8cade9c86c4b59c7db6cbedfc41fac39a811d7f39f4bdeb0d"
|
| 32 |
+
},
|
| 33 |
+
{
|
| 34 |
+
"path": "02_per_label_transfer_metrics.pdf",
|
| 35 |
+
"bytes": 21379,
|
| 36 |
+
"sha256": "10756816e8ad800f5d872907986680abf637e76fe47f9790f27645f8f231ae4a"
|
| 37 |
+
},
|
| 38 |
+
{
|
| 39 |
+
"path": "02_per_label_transfer_metrics.png",
|
| 40 |
+
"bytes": 138298,
|
| 41 |
+
"sha256": "5b558738b405a7b04bb30f38b6e53af21b2392ea12cad0d777e9d6d6985d3d04"
|
| 42 |
+
},
|
| 43 |
+
{
|
| 44 |
+
"path": "03_quality_efficiency.csv",
|
| 45 |
+
"bytes": 448,
|
| 46 |
+
"sha256": "f2a34cbcd700b2663cf761a5fb1dbd3aa99f310a96d4b4477a3a4529bee1a1b3"
|
| 47 |
+
},
|
| 48 |
+
{
|
| 49 |
+
"path": "03_quality_efficiency.pdf",
|
| 50 |
+
"bytes": 21933,
|
| 51 |
+
"sha256": "18c81ce50fc7871b000c9e387f0f7d7780002cb2bd249153aadfa55a186342c0"
|
| 52 |
+
},
|
| 53 |
+
{
|
| 54 |
+
"path": "03_quality_efficiency.png",
|
| 55 |
+
"bytes": 99374,
|
| 56 |
+
"sha256": "08ed7ca8193d5f9065645fb22491ab88ee260d43566e9013aa0f98d7ab4dde87"
|
| 57 |
+
},
|
| 58 |
+
{
|
| 59 |
+
"path": "04_compression_ladder.csv",
|
| 60 |
+
"bytes": 352,
|
| 61 |
+
"sha256": "2dfac2b8330724ce46a22d8f3ddfeffc767f52b93a93cb1a833c89c5820ffccc"
|
| 62 |
+
},
|
| 63 |
+
{
|
| 64 |
+
"path": "04_compression_ladder.pdf",
|
| 65 |
+
"bytes": 19490,
|
| 66 |
+
"sha256": "953d6e356498127b2a607530fac2e41b07b4bf38ed0af945f778b784dc6f7b23"
|
| 67 |
+
},
|
| 68 |
+
{
|
| 69 |
+
"path": "04_compression_ladder.png",
|
| 70 |
+
"bytes": 120809,
|
| 71 |
+
"sha256": "9f6c1d6fdefd45711fcfa57a7d0d91f6baaa3a5ee0b44a36554a9c74febe7090"
|
| 72 |
+
},
|
| 73 |
+
{
|
| 74 |
+
"path": "README.md",
|
| 75 |
+
"bytes": 683,
|
| 76 |
+
"sha256": "7e00107e6576da117689dca0c164c497ce32e9d93c034331e2e161625afc2092"
|
| 77 |
+
},
|
| 78 |
+
{
|
| 79 |
+
"path": "source_summary.json",
|
| 80 |
+
"bytes": 26913,
|
| 81 |
+
"sha256": "3d2c9fcf9c6cc7da1a885968b70a42eaa38659d317c9d3290a2b3066f3f9ee6a"
|
| 82 |
+
}
|
| 83 |
+
]
|
| 84 |
+
}
|
benchmark/figures/source_summary.json
ADDED
|
@@ -0,0 +1,891 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "standalone4l-complete-transfer-benchmark-v2",
|
| 3 |
+
"status": "complete_from_frozen_reports",
|
| 4 |
+
"claim_scope": "Exploratory fresh-head transfer evidence on one English task and a previously opened 55-article SemEval-derived test split.",
|
| 5 |
+
"headless_root_requires_adaptation": true,
|
| 6 |
+
"test_previously_opened": true,
|
| 7 |
+
"test_articles": 55,
|
| 8 |
+
"test_windows": 434,
|
| 9 |
+
"seeds": [
|
| 10 |
+
41,
|
| 11 |
+
42,
|
| 12 |
+
43
|
| 13 |
+
],
|
| 14 |
+
"three_seed_sd_is_not_confidence_interval": true,
|
| 15 |
+
"multilingual_downstream_evaluation_performed": false,
|
| 16 |
+
"upstream_multilingual_metadata": {
|
| 17 |
+
"source": "Qwen/Qwen3.5-0.8B official model card",
|
| 18 |
+
"languages_and_dialects_claimed_upstream": 201,
|
| 19 |
+
"scope": "inherited tokenizer and architecture coverage only"
|
| 20 |
+
},
|
| 21 |
+
"protocol": {
|
| 22 |
+
"epochs": 5,
|
| 23 |
+
"primary_epoch": 5,
|
| 24 |
+
"early_stopping": false,
|
| 25 |
+
"max_length": 256,
|
| 26 |
+
"stride": 128,
|
| 27 |
+
"window_target": "span-aligned technique labels",
|
| 28 |
+
"article_aggregation": "labelwise maximum probability",
|
| 29 |
+
"micro_batch_size": 1,
|
| 30 |
+
"gradient_accumulation": 32,
|
| 31 |
+
"effective_batch_size": 32,
|
| 32 |
+
"learning_rate": 2e-05,
|
| 33 |
+
"weight_decay": 0.01,
|
| 34 |
+
"gradient_clip": 1.0,
|
| 35 |
+
"optimizer": "AdamW",
|
| 36 |
+
"dtype": "bfloat16",
|
| 37 |
+
"thresholds": [
|
| 38 |
+
0.05,
|
| 39 |
+
0.1,
|
| 40 |
+
0.15,
|
| 41 |
+
0.2,
|
| 42 |
+
0.25,
|
| 43 |
+
0.3,
|
| 44 |
+
0.35,
|
| 45 |
+
0.4,
|
| 46 |
+
0.45,
|
| 47 |
+
0.5,
|
| 48 |
+
0.55,
|
| 49 |
+
0.6,
|
| 50 |
+
0.65,
|
| 51 |
+
0.7,
|
| 52 |
+
0.75,
|
| 53 |
+
0.8,
|
| 54 |
+
0.85,
|
| 55 |
+
0.9
|
| 56 |
+
],
|
| 57 |
+
"threshold_selection": "epoch-5 calibration macro-F1, then micro-F1"
|
| 58 |
+
},
|
| 59 |
+
"arms": {
|
| 60 |
+
"task_agnostic_base": {
|
| 61 |
+
"display": "Task-agnostic KD base",
|
| 62 |
+
"lineage": "root base plus a fresh 14-label transfer head",
|
| 63 |
+
"metrics": {
|
| 64 |
+
"macro_f1": {
|
| 65 |
+
"values": [
|
| 66 |
+
0.5883093293517536,
|
| 67 |
+
0.6028379036218096,
|
| 68 |
+
0.5712816197954101
|
| 69 |
+
],
|
| 70 |
+
"mean": 0.5874762842563245,
|
| 71 |
+
"sample_sd": 0.01579462678670608,
|
| 72 |
+
"n": 3
|
| 73 |
+
},
|
| 74 |
+
"micro_f1": {
|
| 75 |
+
"values": [
|
| 76 |
+
0.6468085106382978,
|
| 77 |
+
0.6655172413793103,
|
| 78 |
+
0.6465256797583082
|
| 79 |
+
],
|
| 80 |
+
"mean": 0.6529504772586388,
|
| 81 |
+
"sample_sd": 0.010884055708931178,
|
| 82 |
+
"n": 3
|
| 83 |
+
},
|
| 84 |
+
"exact_match": {
|
| 85 |
+
"values": [
|
| 86 |
+
0.05454545454545454,
|
| 87 |
+
0.03636363636363636,
|
| 88 |
+
0.0
|
| 89 |
+
],
|
| 90 |
+
"mean": 0.0303030303030303,
|
| 91 |
+
"sample_sd": 0.027773186030035393,
|
| 92 |
+
"n": 3
|
| 93 |
+
},
|
| 94 |
+
"predicted_positive_rate": {
|
| 95 |
+
"values": [
|
| 96 |
+
0.5714285714285714,
|
| 97 |
+
0.4090909090909091,
|
| 98 |
+
0.5155844155844156
|
| 99 |
+
],
|
| 100 |
+
"mean": 0.4987012987012987,
|
| 101 |
+
"sample_sd": 0.08247520160679181,
|
| 102 |
+
"n": 3
|
| 103 |
+
}
|
| 104 |
+
},
|
| 105 |
+
"resources": {
|
| 106 |
+
"parameters": {
|
| 107 |
+
"values": [
|
| 108 |
+
334111040.0,
|
| 109 |
+
334111040.0,
|
| 110 |
+
334111040.0
|
| 111 |
+
],
|
| 112 |
+
"mean": 334111040.0,
|
| 113 |
+
"sample_sd": 0.0,
|
| 114 |
+
"n": 3
|
| 115 |
+
},
|
| 116 |
+
"peak_cuda_allocated_bytes": {
|
| 117 |
+
"values": [
|
| 118 |
+
3380975616.0,
|
| 119 |
+
3380975616.0,
|
| 120 |
+
3380975616.0
|
| 121 |
+
],
|
| 122 |
+
"mean": 3380975616.0,
|
| 123 |
+
"sample_sd": 0.0,
|
| 124 |
+
"n": 3
|
| 125 |
+
},
|
| 126 |
+
"peak_cuda_reserved_bytes": {
|
| 127 |
+
"values": [
|
| 128 |
+
3519021056.0,
|
| 129 |
+
3519021056.0,
|
| 130 |
+
3510632448.0
|
| 131 |
+
],
|
| 132 |
+
"mean": 3516224853.3333335,
|
| 133 |
+
"sample_sd": 4843165.086926248,
|
| 134 |
+
"n": 3
|
| 135 |
+
},
|
| 136 |
+
"total_wall_seconds": {
|
| 137 |
+
"values": [
|
| 138 |
+
1583.0297977919981,
|
| 139 |
+
1588.617994428001,
|
| 140 |
+
1603.4310500120046
|
| 141 |
+
],
|
| 142 |
+
"mean": 1591.692947410668,
|
| 143 |
+
"sample_sd": 10.54249851412311,
|
| 144 |
+
"n": 3
|
| 145 |
+
},
|
| 146 |
+
"test_seconds_per_article": {
|
| 147 |
+
"values": [
|
| 148 |
+
0.08164824154545468,
|
| 149 |
+
0.07871871667289683,
|
| 150 |
+
0.08355205721825107
|
| 151 |
+
],
|
| 152 |
+
"mean": 0.08130633847886752,
|
| 153 |
+
"sample_sd": 0.0024347419754560395,
|
| 154 |
+
"n": 3
|
| 155 |
+
}
|
| 156 |
+
},
|
| 157 |
+
"thresholds": [
|
| 158 |
+
0.15,
|
| 159 |
+
0.3,
|
| 160 |
+
0.1
|
| 161 |
+
]
|
| 162 |
+
},
|
| 163 |
+
"structural_copy_control": {
|
| 164 |
+
"display": "Structural copy, no task-free KD",
|
| 165 |
+
"lineage": "same four-layer structure without task-free KD",
|
| 166 |
+
"metrics": {
|
| 167 |
+
"macro_f1": {
|
| 168 |
+
"values": [
|
| 169 |
+
0.5169424579262757,
|
| 170 |
+
0.5349405779540175,
|
| 171 |
+
0.5023778364496152
|
| 172 |
+
],
|
| 173 |
+
"mean": 0.5180869574433028,
|
| 174 |
+
"sample_sd": 0.016311512588628545,
|
| 175 |
+
"n": 3
|
| 176 |
+
},
|
| 177 |
+
"micro_f1": {
|
| 178 |
+
"values": [
|
| 179 |
+
0.5773955773955775,
|
| 180 |
+
0.6083445491251682,
|
| 181 |
+
0.585635359116022
|
| 182 |
+
],
|
| 183 |
+
"mean": 0.5904584952122559,
|
| 184 |
+
"sample_sd": 0.01602830914762526,
|
| 185 |
+
"n": 3
|
| 186 |
+
},
|
| 187 |
+
"exact_match": {
|
| 188 |
+
"values": [
|
| 189 |
+
0.0,
|
| 190 |
+
0.0,
|
| 191 |
+
0.0
|
| 192 |
+
],
|
| 193 |
+
"mean": 0.0,
|
| 194 |
+
"sample_sd": 0.0,
|
| 195 |
+
"n": 3
|
| 196 |
+
},
|
| 197 |
+
"predicted_positive_rate": {
|
| 198 |
+
"values": [
|
| 199 |
+
0.712987012987013,
|
| 200 |
+
0.6207792207792208,
|
| 201 |
+
0.5961038961038961
|
| 202 |
+
],
|
| 203 |
+
"mean": 0.6432900432900432,
|
| 204 |
+
"sample_sd": 0.06160737456395461,
|
| 205 |
+
"n": 3
|
| 206 |
+
}
|
| 207 |
+
},
|
| 208 |
+
"resources": {
|
| 209 |
+
"parameters": {
|
| 210 |
+
"values": [
|
| 211 |
+
334111040.0,
|
| 212 |
+
334111040.0,
|
| 213 |
+
334111040.0
|
| 214 |
+
],
|
| 215 |
+
"mean": 334111040.0,
|
| 216 |
+
"sample_sd": 0.0,
|
| 217 |
+
"n": 3
|
| 218 |
+
},
|
| 219 |
+
"peak_cuda_allocated_bytes": {
|
| 220 |
+
"values": [
|
| 221 |
+
3380975616.0,
|
| 222 |
+
3380975616.0,
|
| 223 |
+
3380975616.0
|
| 224 |
+
],
|
| 225 |
+
"mean": 3380975616.0,
|
| 226 |
+
"sample_sd": 0.0,
|
| 227 |
+
"n": 3
|
| 228 |
+
},
|
| 229 |
+
"peak_cuda_reserved_bytes": {
|
| 230 |
+
"values": [
|
| 231 |
+
3519021056.0,
|
| 232 |
+
3519021056.0,
|
| 233 |
+
3510632448.0
|
| 234 |
+
],
|
| 235 |
+
"mean": 3516224853.3333335,
|
| 236 |
+
"sample_sd": 4843165.086926248,
|
| 237 |
+
"n": 3
|
| 238 |
+
},
|
| 239 |
+
"total_wall_seconds": {
|
| 240 |
+
"values": [
|
| 241 |
+
1575.5905925639963,
|
| 242 |
+
1623.5639530190092,
|
| 243 |
+
1629.195114912989
|
| 244 |
+
],
|
| 245 |
+
"mean": 1609.4498868319981,
|
| 246 |
+
"sample_sd": 29.457874198835384,
|
| 247 |
+
"n": 3
|
| 248 |
+
},
|
| 249 |
+
"test_seconds_per_article": {
|
| 250 |
+
"values": [
|
| 251 |
+
0.07877608058201573,
|
| 252 |
+
0.08307423260041767,
|
| 253 |
+
0.08163602005436339
|
| 254 |
+
],
|
| 255 |
+
"mean": 0.08116211107893226,
|
| 256 |
+
"sample_sd": 0.002187914527825257,
|
| 257 |
+
"n": 3
|
| 258 |
+
}
|
| 259 |
+
},
|
| 260 |
+
"thresholds": [
|
| 261 |
+
0.05,
|
| 262 |
+
0.2,
|
| 263 |
+
0.2
|
| 264 |
+
]
|
| 265 |
+
},
|
| 266 |
+
"existing_specialized_separate_lineage": {
|
| 267 |
+
"display": "Existing specialized 4L, separate lineage",
|
| 268 |
+
"lineage": "historical task-specific shrink/distillation lineage",
|
| 269 |
+
"metrics": {
|
| 270 |
+
"macro_f1": {
|
| 271 |
+
"values": [
|
| 272 |
+
0.5832848452288824,
|
| 273 |
+
0.5958405181160762,
|
| 274 |
+
0.5835785045218137
|
| 275 |
+
],
|
| 276 |
+
"mean": 0.5875679559555907,
|
| 277 |
+
"sample_sd": 0.007165753447433348,
|
| 278 |
+
"n": 3
|
| 279 |
+
},
|
| 280 |
+
"micro_f1": {
|
| 281 |
+
"values": [
|
| 282 |
+
0.6494252873563218,
|
| 283 |
+
0.6458616010854816,
|
| 284 |
+
0.6486486486486486
|
| 285 |
+
],
|
| 286 |
+
"mean": 0.647978512363484,
|
| 287 |
+
"sample_sd": 0.0018739735696546695,
|
| 288 |
+
"n": 3
|
| 289 |
+
},
|
| 290 |
+
"exact_match": {
|
| 291 |
+
"values": [
|
| 292 |
+
0.0,
|
| 293 |
+
0.0,
|
| 294 |
+
0.01818181818181818
|
| 295 |
+
],
|
| 296 |
+
"mean": 0.006060606060606061,
|
| 297 |
+
"sample_sd": 0.01049727762162956,
|
| 298 |
+
"n": 3
|
| 299 |
+
},
|
| 300 |
+
"predicted_positive_rate": {
|
| 301 |
+
"values": [
|
| 302 |
+
0.5597402597402598,
|
| 303 |
+
0.612987012987013,
|
| 304 |
+
0.5688311688311688
|
| 305 |
+
],
|
| 306 |
+
"mean": 0.5805194805194805,
|
| 307 |
+
"sample_sd": 0.02848274311618351,
|
| 308 |
+
"n": 3
|
| 309 |
+
}
|
| 310 |
+
},
|
| 311 |
+
"resources": {
|
| 312 |
+
"parameters": {
|
| 313 |
+
"values": [
|
| 314 |
+
210903360.0,
|
| 315 |
+
210903360.0,
|
| 316 |
+
210903360.0
|
| 317 |
+
],
|
| 318 |
+
"mean": 210903360.0,
|
| 319 |
+
"sample_sd": 0.0,
|
| 320 |
+
"n": 3
|
| 321 |
+
},
|
| 322 |
+
"peak_cuda_allocated_bytes": {
|
| 323 |
+
"values": [
|
| 324 |
+
2143655936.0,
|
| 325 |
+
2143655936.0,
|
| 326 |
+
2143655936.0
|
| 327 |
+
],
|
| 328 |
+
"mean": 2143655936.0,
|
| 329 |
+
"sample_sd": 0.0,
|
| 330 |
+
"n": 3
|
| 331 |
+
},
|
| 332 |
+
"peak_cuda_reserved_bytes": {
|
| 333 |
+
"values": [
|
| 334 |
+
2281701376.0,
|
| 335 |
+
2281701376.0,
|
| 336 |
+
2273312768.0
|
| 337 |
+
],
|
| 338 |
+
"mean": 2278905173.3333335,
|
| 339 |
+
"sample_sd": 4843165.086926248,
|
| 340 |
+
"n": 3
|
| 341 |
+
},
|
| 342 |
+
"total_wall_seconds": {
|
| 343 |
+
"values": [
|
| 344 |
+
1634.488557530014,
|
| 345 |
+
1652.9397718419787,
|
| 346 |
+
1643.488202479988
|
| 347 |
+
],
|
| 348 |
+
"mean": 1643.6388439506602,
|
| 349 |
+
"sample_sd": 9.226529522850198,
|
| 350 |
+
"n": 3
|
| 351 |
+
},
|
| 352 |
+
"test_seconds_per_article": {
|
| 353 |
+
"values": [
|
| 354 |
+
0.0847959352360869,
|
| 355 |
+
0.08598086021798239,
|
| 356 |
+
0.08552987641781908
|
| 357 |
+
],
|
| 358 |
+
"mean": 0.08543555729062946,
|
| 359 |
+
"sample_sd": 0.0005980667826383295,
|
| 360 |
+
"n": 3
|
| 361 |
+
}
|
| 362 |
+
},
|
| 363 |
+
"thresholds": [
|
| 364 |
+
0.1,
|
| 365 |
+
0.05,
|
| 366 |
+
0.05
|
| 367 |
+
]
|
| 368 |
+
}
|
| 369 |
+
},
|
| 370 |
+
"same_seed_macro_f1_deltas_base_minus_control": [
|
| 371 |
+
0.07136687142547793,
|
| 372 |
+
0.06789732566779205,
|
| 373 |
+
0.06890378334579494
|
| 374 |
+
],
|
| 375 |
+
"mean_macro_f1_delta_base_minus_control": 0.06938932681302164,
|
| 376 |
+
"publication_gate_passed": true,
|
| 377 |
+
"per_label_summary": [
|
| 378 |
+
{
|
| 379 |
+
"arm": "task_agnostic_base",
|
| 380 |
+
"label": "Appeal_to_Authority",
|
| 381 |
+
"support": 11,
|
| 382 |
+
"precision_mean": 0.3958614484930274,
|
| 383 |
+
"precision_sample_sd": 0.1238064457010119,
|
| 384 |
+
"recall_mean": 0.5757575757575757,
|
| 385 |
+
"recall_sample_sd": 0.05248638810814781,
|
| 386 |
+
"f1_mean": 0.4657088122605364,
|
| 387 |
+
"f1_sample_sd": 0.1020990126979069
|
| 388 |
+
},
|
| 389 |
+
{
|
| 390 |
+
"arm": "task_agnostic_base",
|
| 391 |
+
"label": "Appeal_to_fear-prejudice",
|
| 392 |
+
"support": 24,
|
| 393 |
+
"precision_mean": 0.5540244550770866,
|
| 394 |
+
"precision_sample_sd": 0.02525673990402998,
|
| 395 |
+
"recall_mean": 0.7361111111111112,
|
| 396 |
+
"recall_sample_sd": 0.13393959390267993,
|
| 397 |
+
"f1_mean": 0.6277521761392729,
|
| 398 |
+
"f1_sample_sd": 0.04994879016183189
|
| 399 |
+
},
|
| 400 |
+
{
|
| 401 |
+
"arm": "task_agnostic_base",
|
| 402 |
+
"label": "Bandwagon,Reductio_ad_hitlerum",
|
| 403 |
+
"support": 7,
|
| 404 |
+
"precision_mean": 0.4851851851851852,
|
| 405 |
+
"precision_sample_sd": 0.07882887649552875,
|
| 406 |
+
"recall_mean": 0.8095238095238095,
|
| 407 |
+
"recall_sample_sd": 0.08247860988423221,
|
| 408 |
+
"f1_mean": 0.6006778309409889,
|
| 409 |
+
"f1_sample_sd": 0.04793776258918466
|
| 410 |
+
},
|
| 411 |
+
{
|
| 412 |
+
"arm": "task_agnostic_base",
|
| 413 |
+
"label": "Black-and-White_Fallacy",
|
| 414 |
+
"support": 12,
|
| 415 |
+
"precision_mean": 0.3667929292929293,
|
| 416 |
+
"precision_sample_sd": 0.07670649392711647,
|
| 417 |
+
"recall_mean": 0.47222222222222227,
|
| 418 |
+
"recall_sample_sd": 0.09622504486493764,
|
| 419 |
+
"f1_mean": 0.4053892966936445,
|
| 420 |
+
"f1_sample_sd": 0.04211369218048746
|
| 421 |
+
},
|
| 422 |
+
{
|
| 423 |
+
"arm": "task_agnostic_base",
|
| 424 |
+
"label": "Causal_Oversimplification",
|
| 425 |
+
"support": 18,
|
| 426 |
+
"precision_mean": 0.5428824049513704,
|
| 427 |
+
"precision_sample_sd": 0.039216350192122156,
|
| 428 |
+
"recall_mean": 0.7962962962962963,
|
| 429 |
+
"recall_sample_sd": 0.11564811108145183,
|
| 430 |
+
"f1_mean": 0.6446992723588468,
|
| 431 |
+
"f1_sample_sd": 0.0634561308339175
|
| 432 |
+
},
|
| 433 |
+
{
|
| 434 |
+
"arm": "task_agnostic_base",
|
| 435 |
+
"label": "Doubt",
|
| 436 |
+
"support": 22,
|
| 437 |
+
"precision_mean": 0.5223832528180354,
|
| 438 |
+
"precision_sample_sd": 0.06742780050556177,
|
| 439 |
+
"recall_mean": 0.9848484848484849,
|
| 440 |
+
"recall_sample_sd": 0.026243194054073875,
|
| 441 |
+
"f1_mean": 0.6802057822343391,
|
| 442 |
+
"f1_sample_sd": 0.04928561754686331
|
| 443 |
+
},
|
| 444 |
+
{
|
| 445 |
+
"arm": "task_agnostic_base",
|
| 446 |
+
"label": "Exaggeration,Minimisation",
|
| 447 |
+
"support": 22,
|
| 448 |
+
"precision_mean": 0.4620811287477954,
|
| 449 |
+
"precision_sample_sd": 0.016801198309751,
|
| 450 |
+
"recall_mean": 0.7727272727272727,
|
| 451 |
+
"recall_sample_sd": 0.15745916432444337,
|
| 452 |
+
"f1_mean": 0.5727040816326531,
|
| 453 |
+
"f1_sample_sd": 0.03645259990419198
|
| 454 |
+
},
|
| 455 |
+
{
|
| 456 |
+
"arm": "task_agnostic_base",
|
| 457 |
+
"label": "Flag-Waving",
|
| 458 |
+
"support": 17,
|
| 459 |
+
"precision_mean": 0.5642642642642642,
|
| 460 |
+
"precision_sample_sd": 0.1478145677656334,
|
| 461 |
+
"recall_mean": 0.8235294117647058,
|
| 462 |
+
"recall_sample_sd": 0.1764705882352941,
|
| 463 |
+
"f1_mean": 0.6464506172839506,
|
| 464 |
+
"f1_sample_sd": 0.035742219952758225
|
| 465 |
+
},
|
| 466 |
+
{
|
| 467 |
+
"arm": "task_agnostic_base",
|
| 468 |
+
"label": "Loaded_Language",
|
| 469 |
+
"support": 45,
|
| 470 |
+
"precision_mean": 0.8649032907793016,
|
| 471 |
+
"precision_sample_sd": 0.046984312428427724,
|
| 472 |
+
"recall_mean": 0.9851851851851852,
|
| 473 |
+
"recall_sample_sd": 0.012830005981991702,
|
| 474 |
+
"f1_mean": 0.9208741623431562,
|
| 475 |
+
"f1_sample_sd": 0.03200926878737141
|
| 476 |
+
},
|
| 477 |
+
{
|
| 478 |
+
"arm": "task_agnostic_base",
|
| 479 |
+
"label": "Name_Calling,Labeling",
|
| 480 |
+
"support": 33,
|
| 481 |
+
"precision_mean": 0.7380952380952381,
|
| 482 |
+
"precision_sample_sd": 0.0047619047619048005,
|
| 483 |
+
"recall_mean": 0.9090909090909091,
|
| 484 |
+
"recall_sample_sd": 0.10925912955951485,
|
| 485 |
+
"f1_mean": 0.8125087983911513,
|
| 486 |
+
"f1_sample_sd": 0.04252971464057227
|
| 487 |
+
},
|
| 488 |
+
{
|
| 489 |
+
"arm": "task_agnostic_base",
|
| 490 |
+
"label": "Repetition",
|
| 491 |
+
"support": 23,
|
| 492 |
+
"precision_mean": 0.5196620583717357,
|
| 493 |
+
"precision_sample_sd": 0.021645909050194662,
|
| 494 |
+
"recall_mean": 0.7536231884057971,
|
| 495 |
+
"recall_sample_sd": 0.06641414050660639,
|
| 496 |
+
"f1_mean": 0.6147520781519571,
|
| 497 |
+
"f1_sample_sd": 0.03505927109416221
|
| 498 |
+
},
|
| 499 |
+
{
|
| 500 |
+
"arm": "task_agnostic_base",
|
| 501 |
+
"label": "Slogans",
|
| 502 |
+
"support": 14,
|
| 503 |
+
"precision_mean": 0.5161361626878869,
|
| 504 |
+
"precision_sample_sd": 0.20305677930281646,
|
| 505 |
+
"recall_mean": 0.7380952380952381,
|
| 506 |
+
"recall_sample_sd": 0.10910894511799614,
|
| 507 |
+
"f1_mean": 0.5834824090638044,
|
| 508 |
+
"f1_sample_sd": 0.09862686289220693
|
| 509 |
+
},
|
| 510 |
+
{
|
| 511 |
+
"arm": "task_agnostic_base",
|
| 512 |
+
"label": "Thought-terminating_Cliches",
|
| 513 |
+
"support": 7,
|
| 514 |
+
"precision_mean": 0.3492063492063492,
|
| 515 |
+
"precision_sample_sd": 0.143516994603374,
|
| 516 |
+
"recall_mean": 0.3333333333333333,
|
| 517 |
+
"recall_sample_sd": 0.08247860988423225,
|
| 518 |
+
"f1_mean": 0.319014319014319,
|
| 519 |
+
"f1_sample_sd": 0.04017590711117064
|
| 520 |
+
},
|
| 521 |
+
{
|
| 522 |
+
"arm": "task_agnostic_base",
|
| 523 |
+
"label": "Whataboutism,Straw_Men,Red_Herring",
|
| 524 |
+
"support": 10,
|
| 525 |
+
"precision_mean": 0.2761904761904762,
|
| 526 |
+
"precision_sample_sd": 0.05302632726504782,
|
| 527 |
+
"recall_mean": 0.5,
|
| 528 |
+
"recall_sample_sd": 0.2645751311064591,
|
| 529 |
+
"f1_mean": 0.330448343079922,
|
| 530 |
+
"f1_sample_sd": 0.021845164720973014
|
| 531 |
+
},
|
| 532 |
+
{
|
| 533 |
+
"arm": "structural_copy_control",
|
| 534 |
+
"label": "Appeal_to_Authority",
|
| 535 |
+
"support": 11,
|
| 536 |
+
"precision_mean": 0.26003086419753085,
|
| 537 |
+
"precision_sample_sd": 0.06281357095350094,
|
| 538 |
+
"recall_mean": 0.5757575757575758,
|
| 539 |
+
"recall_sample_sd": 0.262431940540739,
|
| 540 |
+
"f1_mean": 0.35477582846003897,
|
| 541 |
+
"f1_sample_sd": 0.11479479036518875
|
| 542 |
+
},
|
| 543 |
+
{
|
| 544 |
+
"arm": "structural_copy_control",
|
| 545 |
+
"label": "Appeal_to_fear-prejudice",
|
| 546 |
+
"support": 24,
|
| 547 |
+
"precision_mean": 0.5104813315339631,
|
| 548 |
+
"precision_sample_sd": 0.038427508996734555,
|
| 549 |
+
"recall_mean": 0.8611111111111112,
|
| 550 |
+
"recall_sample_sd": 0.024056261216234387,
|
| 551 |
+
"f1_mean": 0.6401745363877202,
|
| 552 |
+
"f1_sample_sd": 0.029305467347150463
|
| 553 |
+
},
|
| 554 |
+
{
|
| 555 |
+
"arm": "structural_copy_control",
|
| 556 |
+
"label": "Bandwagon,Reductio_ad_hitlerum",
|
| 557 |
+
"support": 7,
|
| 558 |
+
"precision_mean": 0.2876623376623376,
|
| 559 |
+
"precision_sample_sd": 0.061386823077618426,
|
| 560 |
+
"recall_mean": 0.8571428571428571,
|
| 561 |
+
"recall_sample_sd": 0.14285714285714285,
|
| 562 |
+
"f1_mean": 0.4306391777656145,
|
| 563 |
+
"f1_sample_sd": 0.08686392624022733
|
| 564 |
+
},
|
| 565 |
+
{
|
| 566 |
+
"arm": "structural_copy_control",
|
| 567 |
+
"label": "Black-and-White_Fallacy",
|
| 568 |
+
"support": 12,
|
| 569 |
+
"precision_mean": 0.31339031339031337,
|
| 570 |
+
"precision_sample_sd": 0.08301311842925053,
|
| 571 |
+
"recall_mean": 0.5277777777777778,
|
| 572 |
+
"recall_sample_sd": 0.2678791878053599,
|
| 573 |
+
"f1_mean": 0.38778427550357375,
|
| 574 |
+
"f1_sample_sd": 0.13069755090515112
|
| 575 |
+
},
|
| 576 |
+
{
|
| 577 |
+
"arm": "structural_copy_control",
|
| 578 |
+
"label": "Causal_Oversimplification",
|
| 579 |
+
"support": 18,
|
| 580 |
+
"precision_mean": 0.4464201043148412,
|
| 581 |
+
"precision_sample_sd": 0.013538446216471536,
|
| 582 |
+
"recall_mean": 0.9259259259259259,
|
| 583 |
+
"recall_sample_sd": 0.03207501495497923,
|
| 584 |
+
"f1_mean": 0.6023809523809524,
|
| 585 |
+
"f1_sample_sd": 0.018643640071557464
|
| 586 |
+
},
|
| 587 |
+
{
|
| 588 |
+
"arm": "structural_copy_control",
|
| 589 |
+
"label": "Doubt",
|
| 590 |
+
"support": 22,
|
| 591 |
+
"precision_mean": 0.4455314009661836,
|
| 592 |
+
"precision_sample_sd": 0.04067072694857813,
|
| 593 |
+
"recall_mean": 1.0,
|
| 594 |
+
"recall_sample_sd": 0.0,
|
| 595 |
+
"f1_mean": 0.615686274509804,
|
| 596 |
+
"f1_sample_sd": 0.039427200694482675
|
| 597 |
+
},
|
| 598 |
+
{
|
| 599 |
+
"arm": "structural_copy_control",
|
| 600 |
+
"label": "Exaggeration,Minimisation",
|
| 601 |
+
"support": 22,
|
| 602 |
+
"precision_mean": 0.4364176150420584,
|
| 603 |
+
"precision_sample_sd": 0.025093958106097894,
|
| 604 |
+
"recall_mean": 0.8787878787878788,
|
| 605 |
+
"recall_sample_sd": 0.05248638810814775,
|
| 606 |
+
"f1_mean": 0.5829803047194351,
|
| 607 |
+
"f1_sample_sd": 0.030899288356345502
|
| 608 |
+
},
|
| 609 |
+
{
|
| 610 |
+
"arm": "structural_copy_control",
|
| 611 |
+
"label": "Flag-Waving",
|
| 612 |
+
"support": 17,
|
| 613 |
+
"precision_mean": 0.3814814814814815,
|
| 614 |
+
"precision_sample_sd": 0.05010277503136551,
|
| 615 |
+
"recall_mean": 0.803921568627451,
|
| 616 |
+
"recall_sample_sd": 0.17970885078258195,
|
| 617 |
+
"f1_mean": 0.5138595287119652,
|
| 618 |
+
"f1_sample_sd": 0.06400931989326974
|
| 619 |
+
},
|
| 620 |
+
{
|
| 621 |
+
"arm": "structural_copy_control",
|
| 622 |
+
"label": "Loaded_Language",
|
| 623 |
+
"support": 45,
|
| 624 |
+
"precision_mean": 0.8492581841638446,
|
| 625 |
+
"precision_sample_sd": 0.016026591848331686,
|
| 626 |
+
"recall_mean": 1.0,
|
| 627 |
+
"recall_sample_sd": 0.0,
|
| 628 |
+
"f1_mean": 0.9184311025253588,
|
| 629 |
+
"f1_sample_sd": 0.009372233868132662
|
| 630 |
+
},
|
| 631 |
+
{
|
| 632 |
+
"arm": "structural_copy_control",
|
| 633 |
+
"label": "Name_Calling,Labeling",
|
| 634 |
+
"support": 33,
|
| 635 |
+
"precision_mean": 0.6375,
|
| 636 |
+
"precision_sample_sd": 0.03410563654946854,
|
| 637 |
+
"recall_mean": 0.9696969696969697,
|
| 638 |
+
"recall_sample_sd": 0.030303030303030276,
|
| 639 |
+
"f1_mean": 0.7685185185185185,
|
| 640 |
+
"f1_sample_sd": 0.02023900779105559
|
| 641 |
+
},
|
| 642 |
+
{
|
| 643 |
+
"arm": "structural_copy_control",
|
| 644 |
+
"label": "Repetition",
|
| 645 |
+
"support": 23,
|
| 646 |
+
"precision_mean": 0.44863933452168747,
|
| 647 |
+
"precision_sample_sd": 0.01520014564765896,
|
| 648 |
+
"recall_mean": 0.9420289855072463,
|
| 649 |
+
"recall_sample_sd": 0.06641414050660639,
|
| 650 |
+
"f1_mean": 0.607248835423033,
|
| 651 |
+
"f1_sample_sd": 0.019858627551810427
|
| 652 |
+
},
|
| 653 |
+
{
|
| 654 |
+
"arm": "structural_copy_control",
|
| 655 |
+
"label": "Slogans",
|
| 656 |
+
"support": 14,
|
| 657 |
+
"precision_mean": 0.3274318274318274,
|
| 658 |
+
"precision_sample_sd": 0.05434598572773684,
|
| 659 |
+
"recall_mean": 0.4523809523809524,
|
| 660 |
+
"recall_sample_sd": 0.1486904285332952,
|
| 661 |
+
"f1_mean": 0.3715828924162258,
|
| 662 |
+
"f1_sample_sd": 0.07106660236343577
|
| 663 |
+
},
|
| 664 |
+
{
|
| 665 |
+
"arm": "structural_copy_control",
|
| 666 |
+
"label": "Thought-terminating_Cliches",
|
| 667 |
+
"support": 7,
|
| 668 |
+
"precision_mean": 0.1142156862745098,
|
| 669 |
+
"precision_sample_sd": 0.012848374923608973,
|
| 670 |
+
"recall_mean": 0.38095238095238093,
|
| 671 |
+
"recall_sample_sd": 0.1649572197684645,
|
| 672 |
+
"f1_mean": 0.17331433998100665,
|
| 673 |
+
"f1_sample_sd": 0.029065880233243847
|
| 674 |
+
},
|
| 675 |
+
{
|
| 676 |
+
"arm": "structural_copy_control",
|
| 677 |
+
"label": "Whataboutism,Straw_Men,Red_Herring",
|
| 678 |
+
"support": 10,
|
| 679 |
+
"precision_mean": 0.19344598054275472,
|
| 680 |
+
"precision_sample_sd": 0.03373872471827226,
|
| 681 |
+
"recall_mean": 0.5666666666666667,
|
| 682 |
+
"recall_sample_sd": 0.2081665999466133,
|
| 683 |
+
"f1_mean": 0.28584083690299267,
|
| 684 |
+
"f1_sample_sd": 0.06078854830133676
|
| 685 |
+
},
|
| 686 |
+
{
|
| 687 |
+
"arm": "existing_specialized_separate_lineage",
|
| 688 |
+
"label": "Appeal_to_Authority",
|
| 689 |
+
"support": 11,
|
| 690 |
+
"precision_mean": 0.37836021505376344,
|
| 691 |
+
"precision_sample_sd": 0.10546365636105155,
|
| 692 |
+
"recall_mean": 0.6666666666666666,
|
| 693 |
+
"recall_sample_sd": 0.2287828616748712,
|
| 694 |
+
"f1_mean": 0.46885361552028215,
|
| 695 |
+
"f1_sample_sd": 0.09502747629890008
|
| 696 |
+
},
|
| 697 |
+
{
|
| 698 |
+
"arm": "existing_specialized_separate_lineage",
|
| 699 |
+
"label": "Appeal_to_fear-prejudice",
|
| 700 |
+
"support": 24,
|
| 701 |
+
"precision_mean": 0.525708635464733,
|
| 702 |
+
"precision_sample_sd": 0.02235198659615819,
|
| 703 |
+
"recall_mean": 0.8888888888888888,
|
| 704 |
+
"recall_sample_sd": 0.048112522432468774,
|
| 705 |
+
"f1_mean": 0.6599065351235072,
|
| 706 |
+
"f1_sample_sd": 0.015362381542320084
|
| 707 |
+
},
|
| 708 |
+
{
|
| 709 |
+
"arm": "existing_specialized_separate_lineage",
|
| 710 |
+
"label": "Bandwagon,Reductio_ad_hitlerum",
|
| 711 |
+
"support": 7,
|
| 712 |
+
"precision_mean": 0.44485144485144484,
|
| 713 |
+
"precision_sample_sd": 0.016487284575115283,
|
| 714 |
+
"recall_mean": 0.7619047619047619,
|
| 715 |
+
"recall_sample_sd": 0.1649572197684645,
|
| 716 |
+
"f1_mean": 0.5571428571428572,
|
| 717 |
+
"f1_sample_sd": 0.051507875363771265
|
| 718 |
+
},
|
| 719 |
+
{
|
| 720 |
+
"arm": "existing_specialized_separate_lineage",
|
| 721 |
+
"label": "Black-and-White_Fallacy",
|
| 722 |
+
"support": 12,
|
| 723 |
+
"precision_mean": 0.4662644544997486,
|
| 724 |
+
"precision_sample_sd": 0.004538274146363688,
|
| 725 |
+
"recall_mean": 0.5833333333333334,
|
| 726 |
+
"recall_sample_sd": 0.08333333333333331,
|
| 727 |
+
"f1_mean": 0.516747552149851,
|
| 728 |
+
"f1_sample_sd": 0.03589484965697345
|
| 729 |
+
},
|
| 730 |
+
{
|
| 731 |
+
"arm": "existing_specialized_separate_lineage",
|
| 732 |
+
"label": "Causal_Oversimplification",
|
| 733 |
+
"support": 18,
|
| 734 |
+
"precision_mean": 0.4472934472934473,
|
| 735 |
+
"precision_sample_sd": 0.013055771210700413,
|
| 736 |
+
"recall_mean": 0.8333333333333333,
|
| 737 |
+
"recall_sample_sd": 0.1469861839480328,
|
| 738 |
+
"f1_mean": 0.5781794553724379,
|
| 739 |
+
"f1_sample_sd": 0.028407563015813122
|
| 740 |
+
},
|
| 741 |
+
{
|
| 742 |
+
"arm": "existing_specialized_separate_lineage",
|
| 743 |
+
"label": "Doubt",
|
| 744 |
+
"support": 22,
|
| 745 |
+
"precision_mean": 0.5045661001171619,
|
| 746 |
+
"precision_sample_sd": 0.023581174969843577,
|
| 747 |
+
"recall_mean": 1.0,
|
| 748 |
+
"recall_sample_sd": 0.0,
|
| 749 |
+
"f1_mean": 0.6704939668174963,
|
| 750 |
+
"f1_sample_sd": 0.020973132363659372
|
| 751 |
+
},
|
| 752 |
+
{
|
| 753 |
+
"arm": "existing_specialized_separate_lineage",
|
| 754 |
+
"label": "Exaggeration,Minimisation",
|
| 755 |
+
"support": 22,
|
| 756 |
+
"precision_mean": 0.44589793281653745,
|
| 757 |
+
"precision_sample_sd": 0.010987847993006481,
|
| 758 |
+
"recall_mean": 0.9393939393939394,
|
| 759 |
+
"recall_sample_sd": 0.06943296507508849,
|
| 760 |
+
"f1_mean": 0.6043956043956044,
|
| 761 |
+
"f1_sample_sd": 0.022305256187015846
|
| 762 |
+
},
|
| 763 |
+
{
|
| 764 |
+
"arm": "existing_specialized_separate_lineage",
|
| 765 |
+
"label": "Flag-Waving",
|
| 766 |
+
"support": 17,
|
| 767 |
+
"precision_mean": 0.5576388888888889,
|
| 768 |
+
"precision_sample_sd": 0.03705367103320583,
|
| 769 |
+
"recall_mean": 0.8823529411764706,
|
| 770 |
+
"recall_sample_sd": 0.11764705882352944,
|
| 771 |
+
"f1_mean": 0.680769868923179,
|
| 772 |
+
"f1_sample_sd": 0.04164658769406446
|
| 773 |
+
},
|
| 774 |
+
{
|
| 775 |
+
"arm": "existing_specialized_separate_lineage",
|
| 776 |
+
"label": "Loaded_Language",
|
| 777 |
+
"support": 45,
|
| 778 |
+
"precision_mean": 0.8387650085763294,
|
| 779 |
+
"precision_sample_sd": 0.01782556577257849,
|
| 780 |
+
"recall_mean": 1.0,
|
| 781 |
+
"recall_sample_sd": 0.0,
|
| 782 |
+
"f1_mean": 0.9122448979591837,
|
| 783 |
+
"f1_sample_sd": 0.01060439269940129
|
| 784 |
+
},
|
| 785 |
+
{
|
| 786 |
+
"arm": "existing_specialized_separate_lineage",
|
| 787 |
+
"label": "Name_Calling,Labeling",
|
| 788 |
+
"support": 33,
|
| 789 |
+
"precision_mean": 0.6957104190221673,
|
| 790 |
+
"precision_sample_sd": 0.019201490809269257,
|
| 791 |
+
"recall_mean": 0.9696969696969697,
|
| 792 |
+
"recall_sample_sd": 0.030303030303030276,
|
| 793 |
+
"f1_mean": 0.8101049442821594,
|
| 794 |
+
"f1_sample_sd": 0.022092733328802537
|
| 795 |
+
},
|
| 796 |
+
{
|
| 797 |
+
"arm": "existing_specialized_separate_lineage",
|
| 798 |
+
"label": "Repetition",
|
| 799 |
+
"support": 23,
|
| 800 |
+
"precision_mean": 0.4781557067271353,
|
| 801 |
+
"precision_sample_sd": 0.009897987322673213,
|
| 802 |
+
"recall_mean": 0.9420289855072463,
|
| 803 |
+
"recall_sample_sd": 0.06641414050660639,
|
| 804 |
+
"f1_mean": 0.6337774426009719,
|
| 805 |
+
"f1_sample_sd": 0.016444118333485254
|
| 806 |
+
},
|
| 807 |
+
{
|
| 808 |
+
"arm": "existing_specialized_separate_lineage",
|
| 809 |
+
"label": "Slogans",
|
| 810 |
+
"support": 14,
|
| 811 |
+
"precision_mean": 0.4334215167548501,
|
| 812 |
+
"precision_sample_sd": 0.05811581861282867,
|
| 813 |
+
"recall_mean": 0.6904761904761905,
|
| 814 |
+
"recall_sample_sd": 0.1649572197684645,
|
| 815 |
+
"f1_mean": 0.5201316298877274,
|
| 816 |
+
"f1_sample_sd": 0.018567913895656114
|
| 817 |
+
},
|
| 818 |
+
{
|
| 819 |
+
"arm": "existing_specialized_separate_lineage",
|
| 820 |
+
"label": "Thought-terminating_Cliches",
|
| 821 |
+
"support": 7,
|
| 822 |
+
"precision_mean": 0.18446115288220552,
|
| 823 |
+
"precision_sample_sd": 0.03641251252864963,
|
| 824 |
+
"recall_mean": 0.6666666666666666,
|
| 825 |
+
"recall_sample_sd": 0.1649572197684645,
|
| 826 |
+
"f1_mean": 0.2868626868626869,
|
| 827 |
+
"f1_sample_sd": 0.0511620864429468
|
| 828 |
+
},
|
| 829 |
+
{
|
| 830 |
+
"arm": "existing_specialized_separate_lineage",
|
| 831 |
+
"label": "Whataboutism,Straw_Men,Red_Herring",
|
| 832 |
+
"support": 10,
|
| 833 |
+
"precision_mean": 0.2777777777777778,
|
| 834 |
+
"precision_sample_sd": 0.0481125224324688,
|
| 835 |
+
"recall_mean": 0.4,
|
| 836 |
+
"recall_sample_sd": 0.0,
|
| 837 |
+
"f1_mean": 0.32634032634032634,
|
| 838 |
+
"f1_sample_sd": 0.03229931575886014
|
| 839 |
+
}
|
| 840 |
+
],
|
| 841 |
+
"compression_ladder": [
|
| 842 |
+
{
|
| 843 |
+
"name": "24L upstream text teacher",
|
| 844 |
+
"layers": 24,
|
| 845 |
+
"parameters": 752393024,
|
| 846 |
+
"weight_bytes": 1504825632,
|
| 847 |
+
"weight_mib": 1435.1135559082031,
|
| 848 |
+
"weight_sha256": "30783da4651259c26b8a204821ea426b9526431e0c14da0d1c6fa284e7414e5e",
|
| 849 |
+
"kd_stage": "",
|
| 850 |
+
"mean_total_loss": "",
|
| 851 |
+
"mean_interface_loss": "",
|
| 852 |
+
"mean_final_loss": ""
|
| 853 |
+
},
|
| 854 |
+
{
|
| 855 |
+
"name": "8L task-free KD",
|
| 856 |
+
"layers": 8,
|
| 857 |
+
"parameters": 420318144,
|
| 858 |
+
"weight_bytes": 840647880,
|
| 859 |
+
"weight_mib": 801.7042922973633,
|
| 860 |
+
"weight_sha256": "086fbaf9a4838ecde56cd3541c9c3212b22d0c5ee745cf2c1c5e0bb31e0ec22c",
|
| 861 |
+
"kd_stage": "24to8",
|
| 862 |
+
"mean_total_loss": 0.17622653172702485,
|
| 863 |
+
"mean_interface_loss": 0.21562811763578793,
|
| 864 |
+
"mean_final_loss": 0.16626427527853593
|
| 865 |
+
},
|
| 866 |
+
{
|
| 867 |
+
"name": "6L task-free KD",
|
| 868 |
+
"layers": 6,
|
| 869 |
+
"parameters": 377207424,
|
| 870 |
+
"weight_bytes": 754423424,
|
| 871 |
+
"weight_mib": 719.4742431640625,
|
| 872 |
+
"weight_sha256": "492749c20f77a42f6bc993f0927a82217b94f872b7eacf25612f7520c9bc28ce",
|
| 873 |
+
"kd_stage": "8to6",
|
| 874 |
+
"mean_total_loss": 0.03382104352249371,
|
| 875 |
+
"mean_interface_loss": 0.04486069045515251,
|
| 876 |
+
"mean_final_loss": 0.031147060785087888
|
| 877 |
+
},
|
| 878 |
+
{
|
| 879 |
+
"name": "4L task-free KD root",
|
| 880 |
+
"layers": 4,
|
| 881 |
+
"parameters": 334096704,
|
| 882 |
+
"weight_bytes": 668198976,
|
| 883 |
+
"weight_mib": 637.2442016601562,
|
| 884 |
+
"weight_sha256": "2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3",
|
| 885 |
+
"kd_stage": "6to4",
|
| 886 |
+
"mean_total_loss": 0.047020394468859195,
|
| 887 |
+
"mean_interface_loss": 0.07048421185527332,
|
| 888 |
+
"mean_final_loss": 0.043119939065377366
|
| 889 |
+
}
|
| 890 |
+
]
|
| 891 |
+
}
|
release_manifest.json
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
{
|
| 2 |
-
"schema": "qwen35-unified-classification-base-release-
|
| 3 |
"status": "complete",
|
| 4 |
"repo_id": "mp-juuuns/qwen35-standalone4l-classification-base",
|
| 5 |
"old_repo_preserved": {
|
|
@@ -10,6 +10,7 @@
|
|
| 10 |
"root_weight_sha256": "2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3",
|
| 11 |
"root_multilingual_metadata_scope": "inherited tokenizer/architecture coverage; no direct multilingual downstream evaluation",
|
| 12 |
"benchmark_schema": "standalone4l-complete-transfer-benchmark-v2",
|
|
|
|
| 13 |
"nested_specialized_is_separate_lineage": true,
|
| 14 |
"source_data_redistributed": false,
|
| 15 |
"files_before_manifest": [
|
|
@@ -30,8 +31,8 @@
|
|
| 30 |
{
|
| 31 |
"path": "README.md",
|
| 32 |
"role": "root model card",
|
| 33 |
-
"bytes":
|
| 34 |
-
"sha256": "
|
| 35 |
"source": "docs/huggingface/UNIFIED_BASE_RELEASE_MODEL_CARD.md"
|
| 36 |
},
|
| 37 |
{
|
|
@@ -55,6 +56,111 @@
|
|
| 55 |
"sha256": "2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b",
|
| 56 |
"source": "generated"
|
| 57 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 58 |
{
|
| 59 |
"path": "benchmark/per_label_metrics.csv",
|
| 60 |
"role": "per-label metrics by arm and seed",
|
|
|
|
| 1 |
{
|
| 2 |
+
"schema": "qwen35-unified-classification-base-release-v3",
|
| 3 |
"status": "complete",
|
| 4 |
"repo_id": "mp-juuuns/qwen35-standalone4l-classification-base",
|
| 5 |
"old_repo_preserved": {
|
|
|
|
| 10 |
"root_weight_sha256": "2732c616772fe320cdea228ab4554981418b1b2bf615c4183fb1ac8e6e2168d3",
|
| 11 |
"root_multilingual_metadata_scope": "inherited tokenizer/architecture coverage; no direct multilingual downstream evaluation",
|
| 12 |
"benchmark_schema": "standalone4l-complete-transfer-benchmark-v2",
|
| 13 |
+
"figure_schema": "standalone4l-matplotlib-figures-v1",
|
| 14 |
"nested_specialized_is_separate_lineage": true,
|
| 15 |
"source_data_redistributed": false,
|
| 16 |
"files_before_manifest": [
|
|
|
|
| 31 |
{
|
| 32 |
"path": "README.md",
|
| 33 |
"role": "root model card",
|
| 34 |
+
"bytes": 14017,
|
| 35 |
+
"sha256": "49cb7082e65f26835c90131ce7ff05f47ebd77ec9352dad491a0083ede70bd50",
|
| 36 |
"source": "docs/huggingface/UNIFIED_BASE_RELEASE_MODEL_CARD.md"
|
| 37 |
},
|
| 38 |
{
|
|
|
|
| 56 |
"sha256": "2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b",
|
| 57 |
"source": "generated"
|
| 58 |
},
|
| 59 |
+
{
|
| 60 |
+
"path": "benchmark/figures/01_overall_transfer_quality.csv",
|
| 61 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 62 |
+
"bytes": 500,
|
| 63 |
+
"sha256": "769a516e53417e16b61cf30d95a02f6c812c0b1536242cdbf836da779753efbd",
|
| 64 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/01_overall_transfer_quality.csv"
|
| 65 |
+
},
|
| 66 |
+
{
|
| 67 |
+
"path": "benchmark/figures/01_overall_transfer_quality.pdf",
|
| 68 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 69 |
+
"bytes": 18071,
|
| 70 |
+
"sha256": "7221e0d2feb356e4d7f44aa0381e576254f734837e3f54fb0dfb98795ec19ded",
|
| 71 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/01_overall_transfer_quality.pdf"
|
| 72 |
+
},
|
| 73 |
+
{
|
| 74 |
+
"path": "benchmark/figures/01_overall_transfer_quality.png",
|
| 75 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 76 |
+
"bytes": 82127,
|
| 77 |
+
"sha256": "6d35b5964791cfff2701e43831996a9aad7d607305ab3031b0782bed2ffffa66",
|
| 78 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/01_overall_transfer_quality.png"
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"path": "benchmark/figures/02_per_label_transfer_metrics.csv",
|
| 82 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 83 |
+
"bytes": 1169,
|
| 84 |
+
"sha256": "50a3ceb0e3146ba8cade9c86c4b59c7db6cbedfc41fac39a811d7f39f4bdeb0d",
|
| 85 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/02_per_label_transfer_metrics.csv"
|
| 86 |
+
},
|
| 87 |
+
{
|
| 88 |
+
"path": "benchmark/figures/02_per_label_transfer_metrics.pdf",
|
| 89 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 90 |
+
"bytes": 21379,
|
| 91 |
+
"sha256": "10756816e8ad800f5d872907986680abf637e76fe47f9790f27645f8f231ae4a",
|
| 92 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/02_per_label_transfer_metrics.pdf"
|
| 93 |
+
},
|
| 94 |
+
{
|
| 95 |
+
"path": "benchmark/figures/02_per_label_transfer_metrics.png",
|
| 96 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 97 |
+
"bytes": 138298,
|
| 98 |
+
"sha256": "5b558738b405a7b04bb30f38b6e53af21b2392ea12cad0d777e9d6d6985d3d04",
|
| 99 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/02_per_label_transfer_metrics.png"
|
| 100 |
+
},
|
| 101 |
+
{
|
| 102 |
+
"path": "benchmark/figures/03_quality_efficiency.csv",
|
| 103 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 104 |
+
"bytes": 448,
|
| 105 |
+
"sha256": "f2a34cbcd700b2663cf761a5fb1dbd3aa99f310a96d4b4477a3a4529bee1a1b3",
|
| 106 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/03_quality_efficiency.csv"
|
| 107 |
+
},
|
| 108 |
+
{
|
| 109 |
+
"path": "benchmark/figures/03_quality_efficiency.pdf",
|
| 110 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 111 |
+
"bytes": 21933,
|
| 112 |
+
"sha256": "18c81ce50fc7871b000c9e387f0f7d7780002cb2bd249153aadfa55a186342c0",
|
| 113 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/03_quality_efficiency.pdf"
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"path": "benchmark/figures/03_quality_efficiency.png",
|
| 117 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 118 |
+
"bytes": 99374,
|
| 119 |
+
"sha256": "08ed7ca8193d5f9065645fb22491ab88ee260d43566e9013aa0f98d7ab4dde87",
|
| 120 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/03_quality_efficiency.png"
|
| 121 |
+
},
|
| 122 |
+
{
|
| 123 |
+
"path": "benchmark/figures/04_compression_ladder.csv",
|
| 124 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 125 |
+
"bytes": 352,
|
| 126 |
+
"sha256": "2dfac2b8330724ce46a22d8f3ddfeffc767f52b93a93cb1a833c89c5820ffccc",
|
| 127 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/04_compression_ladder.csv"
|
| 128 |
+
},
|
| 129 |
+
{
|
| 130 |
+
"path": "benchmark/figures/04_compression_ladder.pdf",
|
| 131 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 132 |
+
"bytes": 19490,
|
| 133 |
+
"sha256": "953d6e356498127b2a607530fac2e41b07b4bf38ed0af945f778b784dc6f7b23",
|
| 134 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/04_compression_ladder.pdf"
|
| 135 |
+
},
|
| 136 |
+
{
|
| 137 |
+
"path": "benchmark/figures/04_compression_ladder.png",
|
| 138 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 139 |
+
"bytes": 120809,
|
| 140 |
+
"sha256": "9f6c1d6fdefd45711fcfa57a7d0d91f6baaa3a5ee0b44a36554a9c74febe7090",
|
| 141 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/04_compression_ladder.png"
|
| 142 |
+
},
|
| 143 |
+
{
|
| 144 |
+
"path": "benchmark/figures/README.md",
|
| 145 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 146 |
+
"bytes": 683,
|
| 147 |
+
"sha256": "7e00107e6576da117689dca0c164c497ce32e9d93c034331e2e161625afc2092",
|
| 148 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/README.md"
|
| 149 |
+
},
|
| 150 |
+
{
|
| 151 |
+
"path": "benchmark/figures/manifest.json",
|
| 152 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 153 |
+
"bytes": 2697,
|
| 154 |
+
"sha256": "b3e3da2fcf1ae8ea9c88019c289e3d45e7945df655218d5c0d3b905769811f50",
|
| 155 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/manifest.json"
|
| 156 |
+
},
|
| 157 |
+
{
|
| 158 |
+
"path": "benchmark/figures/source_summary.json",
|
| 159 |
+
"role": "Matplotlib benchmark figures and source data",
|
| 160 |
+
"bytes": 26913,
|
| 161 |
+
"sha256": "3d2c9fcf9c6cc7da1a885968b70a42eaa38659d317c9d3290a2b3066f3f9ee6a",
|
| 162 |
+
"source": "results/standalone4l_base/20260731-v1/huggingface_figures_v1/source_summary.json"
|
| 163 |
+
},
|
| 164 |
{
|
| 165 |
"path": "benchmark/per_label_metrics.csv",
|
| 166 |
"role": "per-label metrics by arm and seed",
|