Upload README.md with huggingface_hub

README.md

@@ -6,7 +6,6 @@ tags:
   - robotics
   - droid
   - action-conditioned
-  - from-scratch
 ---
 
 # CAC — from-scratch V-JEPA 2-AC predictor
@@ -21,50 +20,50 @@ frozen V-JEPA 2 ViT-g encoder is not included — use Meta's `vjepa2_ac_vit_gian
 Meta's exact `src.models.ac_predictor.vit_ac_predictor` (from
 [facebookresearch/vjepa2](https://github.com/facebookresearch/vjepa2)): 24
 layers, 1024 dim, 16 heads, ~305M params, RoPE, frame-causal block attention,
-action + state token conditioning (7-dim each). Identical to the shipped
-V-JEPA 2-AC predictor.
+action + state token conditioning (7-dim each). Identical to the shipped AC.
 
-## Training
+## Training (this snapshot)
 
-- 250 iterations, effective batch 256 (micro-batch 64 × 4 grad-accum)
-- LR 7.5e-5 → 4.25e-4 → 0 (WSD; 5% warmup, 25% anneal), AdamW wd 0.04, bf16
-- Loss: L1 teacher-forcing + 2-step autoregressive rollout (a faithful
-  reproduction of Meta's `app/vjepa_droid/train.py` loss)
-- Data: ~2,080 DROID left-cam episodes (~2.6 h footage), 8 frames @ 4 fps, 256 px
-- Hardware: 1× H100 80GB, ~56 minutes
-- Final loss: 1.022 → 0.361 (teacher-forced L1)
+- **2,750 iterations** (toward Meta's 94,500-iter recipe; stopped early to
+  benchmark once held-out val plateaued), effective batch 256, LR 7.5e-5 →
+  4.25e-4 → 0 (WSD), AdamW wd 0.04, bf16, `torch.compile`.
+- Loss: L1 teacher-forcing + 2-step autoregressive rollout (faithful port of
+  Meta's `app/vjepa_droid/train.py`).
+- Data: ~51,000 DROID left-cam episodes (~58 h footage — essentially the full
+  DROID 1.0.1 raw corpus), 8 frames @ 4 fps, 256 px; first 240 held out for eval.
+- Hardware: 1× H100 80GB (GCP spot), ~9 h.
 
 ## Evaluation (240 held-out DROID episodes, frozen ViT-g)
 
-- Mean prediction cosine: **0.740** (vs Meta V-JEPA 2-AC ≈ 0.766 on the same set)
-- Open-loop L1 does not yet beat the frozen-scene baseline — expected for a run
-  on ~4% of Meta's data for ~0.3% of its iterations. The frozen encoder carries
-  most of the prediction quality; this predictor is deliberately undertrained
-  (a pipeline-prover / reproducibility artifact, not a competitive model).
+- Mean prediction cosine: **0.782** vs Meta V-JEPA 2-AC ≈ 0.766 on the same set.
+- Held-out val cosine plateaued ~0.818 during training (encoder-dominated metric,
+  converges fast). Prediction cosine is a lenient metric; open-loop planning
+  quality is the real test — see the planning benchmark in the source repo's
+  `EXPERIMENT.md`.
 
 ## File
 
-`cac_fp32.pt` — `{"predictor": state_dict (fp32, 305M params), "iter": 250,
-"train_args": {...}}`. Loads `strict=True` into `vit_ac_predictor(...)`.
+`cac_fp32.safetensors` — predictor state_dict, fp32, 305M params, 300 tensors.
+Safetensors metadata: `iter`, `architecture`. Loads `strict=True` into
+`vit_ac_predictor(...)`.
 
 ## Load
 
 ```python
 import torch
 from huggingface_hub import hf_hub_download
+from safetensors.torch import load_file
 from src.models.ac_predictor import vit_ac_predictor
 
 predictor = vit_ac_predictor(img_size=(256, 256), patch_size=16, num_frames=64,
                              tubelet_size=2, embed_dim=1408)
-sd = torch.load(hf_hub_download("dobri420/vjepa2-cac", "cac_fp32.pt"),
-                map_location="cpu")["predictor"]
+sd = load_file(hf_hub_download("dobri420/vjepa2-cac", "cac_fp32.safetensors"))
 predictor.load_state_dict(sd, strict=True)
 ```
 
 ## Caveats
 
 - Train/eval split is index-based on DROID's deterministic institution ordering
-  (first 240 episodes held out), so the eval set is institution-concentrated
-  rather than a random sample.
-- Trained on public DROID data using Meta's published recipe; this is an
-  independent reproduction, not affiliated with Meta.
+  (first 240 held out), so the eval set is institution-concentrated, not random.
+- Trained on public DROID data using Meta's published recipe; independent
+  reproduction, not affiliated with Meta.