# 2026-06-01 - Release note for `step_7000` from the WSD early-decay web/wiki run

## Release candidate

- run: `20260530_fresh-gpt2small-lr2e4-bs6-wsd-earlydecay7000-final5e6-webwiki`
- config: `configs/testing/20260530_fresh-gpt2small-lr2e4-bs6-wsd-earlydecay7000-final5e6-webwiki.yaml`
- chosen checkpoint: `/mnt/apps/llm-nanochat/checkpoints/20260530_fresh-gpt2small-lr2e4-bs6-wsd-earlydecay7000-final5e6-webwiki/step_7000.pt`
- intended HF repo: `nazdef/gpt2small-en-it-nanochat-lr2e4-bs6-wsd-earlydecay7000-final5e6-webwiki-step7000`

## Why this checkpoint

This run hit its best saved online validation at `step_7000` and then degraded quickly enough that the stop decision was operationally clear.

Best in-run validation:

- `step_7000`: `validation_loss=3.9247755519`, `validation_perplexity=50.6417103477`

Observed later online validation:

- `step_8000`: `4.1199008982`
- `step_9000`: `4.0582408887`
- `step_10000`: `4.0088127699`

That online signal was then checked with the repo-native CPU benchmark across the requested saved checkpoints `7000/8000/9000/10000`.

## Repo-native benchmark verdict

Suite:

- `configs/eval/20260521_pretrain_minimal_en_it_webwiki_step11000.yaml`

Primary decision metric:

- `val_loss_mixed` lower is better

Benchmark ordering:

1. `step_7000`
   - `val_loss_mixed=5.2158`
   - `val_loss_en=5.0181`
   - `val_loss_it=3.9781`
2. `step_8000`
   - `val_loss_mixed=5.3042`
   - `val_loss_en=5.3609`
   - `val_loss_it=4.2465`
3. `step_9000`
   - `val_loss_mixed=5.5408`
   - `val_loss_en=5.3154`
   - `val_loss_it=4.2196`
4. `step_10000`
   - `val_loss_mixed=5.6087`
   - `val_loss_en=5.3671`
   - `val_loss_it=4.2692`

Winner-only supporting signals:

- `ppl_mixed=184.1616`
- `ppl_en=151.1306`
- `ppl_it=53.4156`
- `loop_rate=0.575`
- `repeated_4gram_rate=0.95`
- `cloze_en_contains=0.02`
- `cloze_it_contains=0.10`

Source/domain losses for the winner:

- `books_en=4.7561`
- `books_it=4.8877`
- `code=8.1805`
- `web_en=5.7991`
- `web_it=6.0342`
- `wiki_en=4.0188`
- `wiki_it=4.0581`

## Best-so-far reading among comparable published checkpoints

Token estimate formula:

- `tokens_seen ~= step * batch_size * grad_accum_steps * sequence_length`
- for these comparable GPT-2-small web/wiki and v5 runs: `6 * 16 * 2500 = 240000` tokens per step

Comparable historical order by the same primary metric `val_loss_mixed`:

1. `20260530_fresh-gpt2small-lr2e4-bs6-wsd-earlydecay7000-final5e6-webwiki` `step_7000`
   - `mixed=5.2158`
   - tokens seen: `~1.68B`
2. `202605211121_fresh-gpt2small-lr2e4-bs6-cosine-webwiki` `step_7000`
   - `mixed=5.3558`
   - tokens seen: `~1.68B`
3. `202605181849_stable-config-recipe-v5-gpt2small-lr2e4-batchmaxpossible-bs6-wsd-fastdecay-webwiki` `step_11000`
   - `mixed=5.3576`
   - tokens seen: `~2.64B`
4. `20260522_fresh-gpt2small-lr1e4-bs6-cosine-webwiki` `step_14000`
   - `mixed=5.4493`
   - tokens seen: `~3.36B`
5. `20260517_stable-config-recipe-v5-gpt2small-lr2e4-batchmaxpossible-bs6-wsd-fastdecay` `step_10000`
   - `mixed=5.4756`
   - tokens seen: `~2.40B`
6. `20260525_fresh-gpt2small-lr1e4-bs6-cosine-webwiki` `step_23000`
   - `mixed=5.6642`
   - tokens seen: `~5.52B`

Operational reading:

- this `step_7000` is currently the best checkpoint so far among the comparable benchmarked public GPT-2-small EN/IT checkpoints in this family
- the gain is not magic, but it is real: it beats the previous best public `7k` cosine release while using the same rough token budget
- the run still collapses later, so the win is "best preserved early checkpoint", not "schedule solved forever"