tensorlink-dev/tsfm-toto2-4m-tokenizer-ablation

TSFM tokenizer ablation — Toto-2-4m-recipe clone (v10)

Context-tokenizer ablation on a ~3.6M-param decoder-only patched transformer trained on a TempoPFN-style synthetic prior ensemble with contiguous patch masking, a 9-level quantile head, NorMuon+AdamW, and index RoPE at time-scaled positions. Horizon decoding is fixed patch-32 in every arm; only the CONTEXT tokenizer varies:

arm	tokenizer	history	ctx tokens
T0	fixed-32 (control)	4,096	128
T1	pyramid, iso-context	4,096	44
T2	pyramid, iso-token	16,384	128
T3	adaptive equal-surprise	16,384	128

Each subfolder is one (arm, seed) run: model.pt (final), ckpt_15000.pt (rank-stability snapshot), config.json, results.json (dev-GIFT CRPS + long-horizon probe). Dev metrics use a fixed 14-task GIFT-Eval subset — NOT the full leaderboard; treat numbers as ablation-internal, not comparable to published GIFT scores. Generated by the v10 experiment notebook.

Results

             gm_crps_mean  gm_crps_std  gm_crps_15k  long_season   other  probe_r  probe_amp  n
arm                                                                                            
T0_fixed           0.1949       0.0113       0.1875       0.2570  0.1352  -0.0067     0.2647  3
T1_pyr_ctx         0.2070       0.0089       0.2015       0.2673  0.1473  -0.0020     0.3043  3
T2_pyr_tok         0.2166       0.0143       0.2030       0.2664  0.1647   0.0000     0.2563  3
T3_adaptive        0.2324       0.0145       0.2175       0.2588  0.2021   0.0057     0.2533  3

--- tripwire checks ---
seed noise floor (max arm sigma): 0.0145
  T0_fixed vs T1_pyr_ctx: gap=0.0121 -> UNRESOLVED (< 2*sigma) — add seeds/steps
  T1_pyr_ctx vs T2_pyr_tok: gap=0.0095 -> UNRESOLVED (< 2*sigma) — add seeds/steps
  T2_pyr_tok vs T3_adaptive: gap=0.0158 -> UNRESOLVED (< 2*sigma) — add seeds/steps
  rank stable 15k->30k: ['T0_fixed', 'T1_pyr_ctx', 'T2_pyr_tok', 'T3_adaptive']
  H1 compression tax (T1 vs T0): +6.23% (FAILS H1 gate — read T2 as gain minus tax)