Hugging Face's logo

Buckets:
Eishaan
/
repro-self-training-denoising-forgetting-artifacts

Files
xet
Eishaan/repro-self-training-denoising-forgetting-artifacts / repro-bundle
445 kB
44 files
Updated 28 days ago
Name
Size
outputs
src
README.md2.16 kB
xet
cifar_selfdistill.py7.76 kB
xet
claim1_decomposition.py2.99 kB
xet
claim2_ushape.py3.11 kB
xet
claim3_spectral.py4.55 kB
xet
claim4_igcv.py2.99 kB
xet
general_cov.py2.54 kB
xet
paper.txt96.6 kB
xet
plot_cifar.py1.27 kB
xet
pyproject.toml252 Bytes
xet
uv.lock121 kB
xet
README.md

Reproduction — Why Self-Training Helps and Hurts: Denoising vs. Signal Forgetting

Independent reproduction of ICML 2026 paper VnA5q5jXVz (arXiv 2602.14029), Wu, Yang & Sun. No official code was released; everything here is written from the paper.

What is verified

Claim Content Status Evidence
1 Deterministic-equivalent recursion R*_t = B*_t (forgetting, up) + V*_t (noise, down exp) verified claim1_decomposition.py, general_cov.py (spiked Thm 3.2 and general Thm 4.2)
2 U-shaped risk & optimal early stopping, strictly under anisotropy (s>1) verified claim2_ushape.py (Fig 1b, Fig 3a)
3 Direction-dependent spectral filter: survival (s/(s+tau))^{t+1}, noise (1+tau)^{-t} verified claim3_spectral.py
4 iGCV consistently estimates risk & recovers t* with no validation set verified claim4_igcv.py
5 Trade-off persists for deep nets (ResNet-50 / CIFAR-10 self-distillation) verified (GPU Job) cifar_selfdistill.py (Fig 6a)

Layout

  • src/linear_selftrain.py — Algorithm 1 (ridgeless/ridge self-training), spiked Thm 3.2, multi-spike Thm 3.6, general deterministic-equivalent recursion (Def 4.1 / eq 10), iGCV (eq 11-12), fast structured samplers.
  • src/plotting.py — Plotly + CSV export helpers.
  • claim{1,2,3,4}_*.py, general_cov.py — per-claim reproductions (linear theory, CPU).
  • cifar_selfdistill.py — PEP-723 UV script for the ResNet-50/CIFAR-10 experiment (HF GPU Job).
  • outputs/ — generated figures (HTML) + raw data (CSV) per claim.

Reproduce 

# linear-theory claims (CPU, ~1 min each). WSL2 note: single-thread BLAS is faster.
uv run --env-file .env python claim1_decomposition.py   # .env pins OPENBLAS_NUM_THREADS=1
uv run --env-file .env python claim2_ushape.py
uv run --env-file .env python claim3_spectral.py
uv run --env-file .env python claim4_igcv.py
uv run --env-file .env python general_cov.py

# deep-net claim (GPU): run on Hugging Face Jobs
hf jobs uv run --flavor a10g-small --secrets HF_TOKEN cifar_selfdistill.py \
  --n 10000 --K 4 --epochs 40 --etas 0.4,0.6,0.8
Total size
445 kB
Files
44
Last updated
Jul 17
Pre-warmed CDN
US EU US EU

Contributors