Upload TASK.md with huggingface_hub

TASK.md

@@ -0,0 +1,31 @@
+# TASK — 100M-active MoE, from scratch, physics-sim next-frame prediction, custom minimal vocab
+
+Train a Qwen3-style sparse-MoE LM **from scratch** on the physics-simulation
+next-frame-prediction corpus, using a **custom minimal tokenizer** whose vocab
+contains only the tokens needed to emit the simulation text (digits, punctuation,
+structural keywords). Target ~100M active params.
+
+## Scaffold
+- Model/trainer: github.com/AlexWortega/moe-200m-qwen3-100b- (Qwen3-MoE: GQA + partial RoPE
+  + QK-Norm + RMSNorm, aux-loss-free sigmoid bias router, 1 shared + N routed top-2 SwiGLU
+  experts, tied embed/lm_head, Liger fused-CE, Muon optimizer). `MoEModelConfig` in model.py.
+- Sibling 100M config exists: moe-100m-volta-week (good sizing reference).
+
+## Data (HF Hub, from the physics-llm project)
+- AlexWortega/physics-scenarios-raw, AlexWortega/physics-scenarios-packed (~900K scenes,
+  30 types, 24 train / 6 held-out). Format = the LFM2 serialization (Scene/Gravity/Frame/obj_...).
+
+## Key requirement — custom vocab
+Vocab = ONLY simulation tokens (tens–low-hundreds). With tied embeddings, shrinking vocab from
+151,936 → ~100 frees ~97M embedding params, so the whole ~100M budget goes to the MoE/dynamics
+(vs the 350M LFM2 whose huge vocab embeddings ate the budget). Drop free-text Scene/Frame
+descriptions (not needed for physics); keep Type as a categorical token.
+
+## Success metric
+Pymunk position error as % of scene diagonal (same as LFM2 baseline), via the existing harness
+at /Users/aleksandrnikolich/Desktop/vae_llm/physics_blog/bench (physics_core.rollout). Baseline
+to beat: LFM2-350M bf16 — @15f trained 0.38% / held-out 0.93%; orbit 0.75% @80f.
+
+## High-impact unknowns -> Clarify
+- experiment budget (GPU-h / wall-clock)  - GPU choice (eva02 A6000 vs eva01 4xV100)
+- tokenizer/number encoding (char-level vs tiny-BPE)  [genuine 2-path fork]