| --- |
| license: apache-2.0 |
| library_name: pytorch |
| pipeline_tag: time-series-forecasting |
| tags: |
| - time-series |
| - forecasting |
| - time-series-forecasting |
| - knowledge-distillation |
| - foundation-model |
| - probabilistic-forecasting |
| - gift-eval |
| --- |
| |
| # TimeTron-v2-33M |
|
|
| A **33.5M-parameter** time-series foundation model, distilled from **Chronos-2** by |
| **latent-space knowledge distillation** — the student matches the teacher's internal |
| representations rather than its forecasts. |
|
|
| **GIFT-Eval: 0.8794 normalized MASE / 0.6122 normalized CRPS** across all 97 configurations |
| (official Salesforce harness). On probabilistic accuracy it beats models up to **21× larger**. |
|
|
| | model | params | nMASE | nCRPS | |
| |---|---|---|---| |
| | **TimeTron-v2-33M** | **33.5M** | **0.8794** | **0.6122** | |
| | Chronos-Small | 46M | 0.8917 | 0.6630 | |
| | Moirai-Base | 91M | 0.9010 | 0.6095 | |
| | Chronos-Base | 200M | 0.8758 | 0.6521 | |
| | Chronos-Large | 710M | 0.8696 | 0.6473 | |
| | *Chronos-2 (teacher)* | *~120M* | *0.6978* | *0.4854* | |
|
|
| Trained on **8.25B points** (7.5B latent distillation + 750M output distillation) on a single |
| rented RTX 4090. |
|
|
| ## Usage |
|
|
| ```python |
| import numpy as np |
| from modeling_timetron import TimeTron |
| |
| model = TimeTron.from_pretrained("CorteriIntelligence/TimeTron-v2-33M") |
| |
| context = np.random.randn(4, 512).astype("float32") # (batch, history) |
| quantiles = model.predict(context, prediction_length=128) # (4, 128, 21), input's own scale |
| median = model.predict_median(context, prediction_length=128) # (4, 128) |
| ``` |
|
|
| - **21 quantile levels**: `[0.01, 0.05, 0.1, 0.15 … 0.9, 0.95, 0.99]` — index 10 is the median |
| - **384 steps decoded natively**; longer horizons extend autoregressively in whole chunks |
| - **Any context length** — cropped to a multiple of 32 and left-padded as needed, up to 2048 |
| - Output is in the **input's own scale**; no manual normalization required |
|
|
| Requires `torch`, `numpy`, and `safetensors`. `modeling_timetron.py` is self-contained. |
|
|
| ## Model details |
|
|
| | | | |
| |---|---| |
| | parameters | 33.52M | |
| | hidden / layers / heads | 512 / 12 / 16 | |
| | patch length | 32 | |
| | max context | 2048 (trained at ≤512) | |
| | native horizon | 384 | |
| | attention | bidirectional, RoPE + QK-norm | |
| | normalization | causal patch-norm, asinh-compressed, μ-anchored | |
| | output | 21 quantiles per future patch | |
| | teacher | Chronos-2 (~120M) | |
|
|
| Forecasts are produced in `z = asinh((y − μ)/σ)` space and inverted as `ŷ = μ + σ·sinh(z)`, |
| which is why the model handles wide dynamic ranges without manual scaling. |
|
|
| ### Training |
|
|
| Latent KD at three taps (student layers 4/8/12 → teacher layers 4/8/final) with SmoothL1 |
| against LayerNorm'd, pooled teacher representations, plus a pinball loss on 21 quantiles and a |
| masked-view consistency term. The final checkpoint is a weight average of one latent-distilled |
| checkpoint and two output-distilled variants (`W_OUT` 0.2 and 0.5) — averaging over |
| *objective-diverse* checkpoints was the single most reliable source of gain in the project. |
|
|
| ## Limitations |
|
|
| - **Test-data leakage: declared `Yes`.** Training pools contain series that are GIFT-Eval test |
| datasets (m4 family, LOOP_SEATTLE, SZ_TAXI, hierarchical_sales, restaurant, |
| temperature_rain). Long series used a 10% tail holdout and short series a last-window |
| holdout, but partial overlap with official test horizons remains possible. We declare it |
| rather than argue the edge case. |
| - **Univariate.** The architecture contains a group-attention branch for in-context learning |
| across related series, and `predict(..., group_ids=...)` exposes it — but **on this |
| checkpoint it is untrained and passing `group_ids` is a no-op**. Training it on a frozen |
| backbone measured **3.4% worse** on GIFT's 43 multivariate configurations; the capability |
| turned out to be inseparable from the backbone it co-adapts with. |
| - **Known-future covariates** are supported by the architecture (`future_values`) with zero |
| additional parameters, but are only lightly trained (~5% of rows). |
| - **Trained at context ≤512.** Longer contexts are accepted but untested. |
| - **Horizons beyond 384** use autoregressive chunk extension, not a native long-horizon head. |
|
|
| ## Evaluation |
|
|
| Numbers above are the official GIFT-Eval harness (gluonts metric engine, official windowing |
| and seasonality), aggregated as the geometric mean of per-configuration |
| `model_MASE / seasonal_naive_MASE`. Submission artifacts are in `gifteval_submission/`. |
|
|
| On **IEX** Indian electricity spot prices (private data, clean for every model compared), |
| TimeTron scores **1.189 MASE / 862 CRPS** versus TimesFM-2.5's 1.232 / 908 — better on both at |
| 1/7 the size. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{srivastava2026timetron, |
| title = {TimeTron: Latent-Space Distillation of a Time-Series Foundation Model at 33M Parameters}, |
| author = {Srivastava, Aditya}, |
| year = {2026}, |
| note = {Corteri Intelligence} |
| } |
| ``` |
|
|
| ## Acknowledgements |
|
|
| Distilled from [amazon/chronos-2](https://huggingface.co/amazon/chronos-2). Evaluated with |
| [GIFT-Eval](https://huggingface.co/datasets/Salesforce/GiftEval). Pretraining corpora: LOTSA, |
| Time-300B, GIFT-Eval-Pretrain. |
|
|
