Upload data/pipeline_v2_log.txt with huggingface_hub

data/pipeline_v2_log.txt

@@ -0,0 +1,133 @@
+
+--- streaming: skill_based_easy ---
+  skill_based_easy: scanned=  2000 accepted=   284 (14.2%)
+  skill_based_easy: scanned=  4000 accepted=   562 (14.1%)
+  skill_based_easy: scanned=  6000 accepted=   860 (14.3%)
+  skill_based_easy: scanned=  8000 accepted=  1172 (14.6%)
+  skill_based_easy: scanned= 10000 accepted=  1471 (14.7%)
+  skill_based_easy: scanned= 12000 accepted=  1755 (14.6%)
+  skill_based_easy: scanned= 14000 accepted=  2063 (14.7%)
+  skill_based_easy: scanned= 16000 accepted=  2273 (14.2%)
+  skill_based_easy: scanned= 18000 accepted=  2398 (13.3%)
+  skill_based_easy: scanned= 20000 accepted=  2541 (12.7%)
+  skill_based_easy: scanned= 22000 accepted=  2742 (12.5%)
+  skill_based_easy: scanned= 24000 accepted=  2928 (12.2%)
+  skill_based_easy: scanned= 26000 accepted=  3139 (12.1%)
+  skill_based_easy: scanned= 28000 accepted=  3385 (12.1%)
+  skill_based_easy: scanned= 30000 accepted=  3612 (12.0%)
+  skill_based_easy: scanned= 32000 accepted=  3759 (11.7%)
+  skill_based_easy: scanned= 34000 accepted=  3893 (11.5%)
+  skill_based_easy: scanned= 36000 accepted=  4052 (11.3%)
+  skill_based_easy: scanned= 38000 accepted=  4192 (11.0%)
+  skill_based_easy: scanned= 40000 accepted=  4337 (10.8%)
+  skill_based_easy: scanned= 42000 accepted=  4521 (10.8%)
+  skill_based_easy: scanned= 44000 accepted=  4718 (10.7%)
+  skill_based_easy: DONE scanned=44809 accepted=4804 (10.7%)
+
+--- streaming: skill_based_medium ---
+  skill_based_medium: scanned=  2000 accepted=    84 (4.2%)
+  skill_based_medium: scanned=  4000 accepted=   185 (4.6%)
+  skill_based_medium: scanned=  6000 accepted=   308 (5.1%)
+  skill_based_medium: scanned=  8000 accepted=   420 (5.2%)
+  skill_based_medium: scanned= 10000 accepted=   642 (6.4%)
+  skill_based_medium: scanned= 12000 accepted=   848 (7.1%)
+  skill_based_medium: scanned= 14000 accepted=  1008 (7.2%)
+  skill_based_medium: scanned= 16000 accepted=  1157 (7.2%)
+  skill_based_medium: scanned= 18000 accepted=  1318 (7.3%)
+  skill_based_medium: scanned= 20000 accepted=  1456 (7.3%)
+  skill_based_medium: scanned= 22000 accepted=  1553 (7.1%)
+  skill_based_medium: scanned= 24000 accepted=  1656 (6.9%)
+  skill_based_medium: scanned= 26000 accepted=  1717 (6.6%)
+  skill_based_medium: scanned= 28000 accepted=  1757 (6.3%)
+  skill_based_medium: scanned= 30000 accepted=  1819 (6.1%)
+  skill_based_medium: scanned= 32000 accepted=  1974 (6.2%)
+  skill_based_medium: scanned= 34000 accepted=  2238 (6.6%)
+  skill_based_medium: scanned= 36000 accepted=  2467 (6.9%)
+  skill_based_medium: scanned= 38000 accepted=  2594 (6.8%)
+  skill_based_medium: scanned= 40000 accepted=  2728 (6.8%)
+  skill_based_medium: scanned= 42000 accepted=  2866 (6.8%)
+  skill_based_medium: scanned= 44000 accepted=  3045 (6.9%)
+  skill_based_medium: scanned= 46000 accepted=  3207 (7.0%)
+  skill_based_medium: scanned= 48000 accepted=  3327 (6.9%)
+  skill_based_medium: scanned= 50000 accepted=  3413 (6.8%)
+  skill_based_medium: scanned= 52000 accepted=  3513 (6.8%)
+  skill_based_medium: scanned= 54000 accepted=  3623 (6.7%)
+  skill_based_medium: scanned= 56000 accepted=  3737 (6.7%)
+  skill_based_medium: scanned= 58000 accepted=  3854 (6.6%)
+  skill_based_medium: scanned= 60000 accepted=  3964 (6.6%)
+  skill_based_medium: scanned= 62000 accepted=  4025 (6.5%)
+  skill_based_medium: scanned= 64000 accepted=  4082 (6.4%)
+  skill_based_medium: scanned= 66000 accepted=  4135 (6.3%)
+  skill_based_medium: scanned= 68000 accepted=  4201 (6.2%)
+  skill_based_medium: scanned= 70000 accepted=  4249 (6.1%)
+  skill_based_medium: scanned= 72000 accepted=  4328 (6.0%)
+  skill_based_medium: scanned= 74000 accepted=  4431 (6.0%)
+  skill_based_medium: scanned= 76000 accepted=  4505 (5.9%)
+  skill_based_medium: scanned= 78000 accepted=  4562 (5.8%)
+  skill_based_medium: scanned= 80000 accepted=  4630 (5.8%)
+  skill_based_medium: scanned= 82000 accepted=  4718 (5.8%)
+  skill_based_medium: scanned= 84000 accepted=  4780 (5.7%)
+  skill_based_medium: scanned= 86000 accepted=  4831 (5.6%)
+  skill_based_medium: scanned= 88000 accepted=  4890 (5.6%)
+  skill_based_medium: DONE scanned=89343 accepted=4916 (5.5%)
+
+--- streaming: skill_based_mixed ---
+  skill_based_mixed: scanned=  2000 accepted=    88 (4.4%)
+  skill_based_mixed: scanned=  4000 accepted=   172 (4.3%)
+  skill_based_mixed: DONE scanned=5689 accepted=191 (3.4%)
+
+--- adapter: dataset_adapters/code.parquet ---
+  downloading dataset_adapters/code.parquet (cached if present)...
+  dataset_adapters/code.parquet: scanned=  2000 accepted=   798 (39.9%)
+  dataset_adapters/code.parquet: scanned=  4000 accepted=  1549 (38.7%)
+  dataset_adapters/code.parquet: scanned=  6000 accepted=  2303 (38.4%)
+  dataset_adapters/code.parquet: scanned=  8000 accepted=  3052 (38.1%)
+  dataset_adapters/code.parquet: scanned= 10000 accepted=  3793 (37.9%)
+  dataset_adapters/code.parquet: scanned= 12000 accepted=  4586 (38.2%)
+  dataset_adapters/code.parquet: scanned= 14000 accepted=  5381 (38.4%)
+  dataset_adapters/code.parquet: scanned= 16000 accepted=  6124 (38.3%)
+  dataset_adapters/code.parquet: scanned= 18000 accepted=  6886 (38.3%)
+  dataset_adapters/code.parquet: scanned= 20000 accepted=  7744 (38.7%)
+  dataset_adapters/code.parquet: scanned= 22000 accepted=  8596 (39.1%)
+  dataset_adapters/code.parquet: scanned= 24000 accepted=  9473 (39.5%)
+  dataset_adapters/code.parquet: scanned= 26000 accepted= 10339 (39.8%)
+  dataset_adapters/code.parquet: scanned= 28000 accepted= 11184 (39.9%)
+  dataset_adapters/code.parquet: scanned= 30000 accepted= 12050 (40.2%)
+  dataset_adapters/code.parquet: DONE scanned=31960 accepted=12875 (40.3%)
+
+--- adapter: dataset_adapters/swe.parquet ---
+  downloading dataset_adapters/swe.parquet (cached if present)...
+  dataset_adapters/swe.parquet: scanned=  2000 accepted=   207 (10.3%)
+  dataset_adapters/swe.parquet: scanned=  4000 accepted=   415 (10.4%)
+  dataset_adapters/swe.parquet: scanned=  6000 accepted=   625 (10.4%)
+  dataset_adapters/swe.parquet: scanned=  8000 accepted=   834 (10.4%)
+  dataset_adapters/swe.parquet: scanned= 10000 accepted=  1017 (10.2%)
+  dataset_adapters/swe.parquet: scanned= 12000 accepted=  1291 (10.8%)
+  dataset_adapters/swe.parquet: scanned= 14000 accepted=  1526 (10.9%)
+  dataset_adapters/swe.parquet: scanned= 16000 accepted=  1791 (11.2%)
+  dataset_adapters/swe.parquet: scanned= 18000 accepted=  2067 (11.5%)
+  dataset_adapters/swe.parquet: scanned= 20000 accepted=  2325 (11.6%)
+  dataset_adapters/swe.parquet: scanned= 22000 accepted=  2590 (11.8%)
+  dataset_adapters/swe.parquet: scanned= 24000 accepted=  2823 (11.8%)
+  dataset_adapters/swe.parquet: scanned= 26000 accepted=  3087 (11.9%)
+  dataset_adapters/swe.parquet: scanned= 28000 accepted=  3337 (11.9%)
+  dataset_adapters/swe.parquet: scanned= 30000 accepted=  3614 (12.0%)
+  dataset_adapters/swe.parquet: DONE scanned=31661 accepted=3841 (12.1%)
+
+
+========== FINAL ==========
+Output: data\microagent_train_v2.jsonl
+Total scanned : 203462
+Total accepted: 26627  (13.1%)
+
+Per-source accepted:
+  dataset_adapters/code.parquet                       12875 /  31960  ( 40.3%)
+  skill_based_medium                                   4916 /  89343  (  5.5%)
+  skill_based_easy                                     4804 /  44809  ( 10.7%)
+  dataset_adapters/swe.parquet                         3841 /  31661  ( 12.1%)
+  skill_based_mixed                                     191 /   5689  (  3.4%)
+
+Code (v2) breakdown:
+  success/<finish>: 11529
+  give_up:          1346
+  rejected reasons: too_few=487  too_many=103  early_parse_err=20  json_unrecoverable=15833  no_real_attempt=2642  per_task_cap=0