EgoWM Lane B โ Wan2.1-T2V-1.3B + cam/hand heads (frozen trunk)
Project: github.com/geng-haoran/EgoWM
This ckpt is Lane B's first real-finetune-at-scale on a single GH200: the Wan2.1-T2V-1.3B DiT trunk is frozen and 8.37 M trainable parameters on the camera + hand regression heads are tuned with rectified-flow latents cached from 1,873 ARCTIC clips.
Held-out evaluation (24-clip val pool, ARCTIC)
| metric | this ckpt | predict-train-mean baseline | ฮ |
|---|---|---|---|
| Hand 21-keypoint MPJPE (mm) | 195.8 | 207.3 | -5.5 % |
| Hand wrist translation MAE (mm) | 181.8 | 178.1 | +2.1 % |
| Camera translation MAE (cm) | 7.53 | 3.64 | +107.2 % |
Training details: 2000 steps ร 1 step/s on the cached-latent ARCTIC pool;
lr 5e-4; AdamW. Loss curves + per-ckpt eval in the repo (_viz/21_train_curve.png,
_viz/30_final_summary.png).
Loading
import torch
from egowm.models.egowm_toy import EgoWMToy
from egowm.models.wan_load import load_pretrained_wan
m = EgoWMToy(z_rgb=16, cond_ch=17, dim=1536, ffn_dim=8960, layers=30,
heads=12, model_type="t2v", cam_dim=7, hand_dim=61, n_hands=2)
load_pretrained_wan(m.dit, "Wan2.1-T2V-1.3B/diffusion_pytorch_model.safetensors")
sd = torch.load("ckpt_step001200.pt", weights_only=False)["state_dict"]
m.load_state_dict(sd)
Manifest
Trained on: data_engine/manifests/train_egowm_arctic.jsonl (1873 ARCTIC clips with both DA3 metric geometry and Wan2.1-VAE latents cached on disk).
Lane B / Lane A handoff
This was produced by h200_1 (Lane B) of the EgoWM multi-machine sprint. See
docs/WORK_CLAIMS.md + docs/17_lane_b_e2e_report.md for the full pipeline,
HaWoR auto-label outputs, dataset deep-dive PPT, and remaining steps.