| --- |
| |
| |
| |
| |
| pipeline_tag: robotics |
| license: gemma |
| datasets: |
| - allenai/MolmoAct2-BimanualYAM-Dataset |
| tags: |
| - robotics |
| - vla |
| - openpi |
| - pi05 |
| - bimanual |
| - yam |
| - jax |
| - lerobot |
| inference: false |
| model-index: |
| - name: pi05-yam-molmoact2 |
| results: |
| - task: |
| type: robotics |
| dataset: |
| type: allenai/19012026-block-13 |
| name: held-out YAM episode repo (open-loop) |
| metrics: |
| - type: open_loop_mse |
| name: open-loop action MSE (16-step chunks, n=20) |
| value: 0.00206 |
| --- |
| |
| # π0.5: YAM bimanual fine-tune on MolmoAct2 data |
|
|
| [](https://github.com/robocurve/inspect-robots) |
| [](https://github.com/robocurve/inspect-robots-yam) |
| [](https://worldevals.org) |
| [](https://huggingface.co/collections/robocurve/worldpolicies-6a4dc8fd556a82aeea0fca37) |
|
|
| JAX/Orbax fine-tune of [openpi](https://github.com/Physical-Intelligence/openpi) |
| `pi05_base` (full fine-tune, nothing frozen; PaliGemma backbone → Gemma license terms) |
| for the bimanual YAM platform, trained on the AllenAI |
| [MolmoAct2-BimanualYAM dataset](https://huggingface.co/datasets/allenai/MolmoAct2-BimanualYAM-Dataset) |
| (124 LeRobot repos, 5,145 episodes, 11.35M frames). Independent replication of a reference |
| fine-tune on the same data; full recipe at |
| [robocurve/pi05-yam-replication](https://github.com/robocurve/pi05-yam-replication). |
|
|
| ## Intended use & safety |
|
|
| - **Intended use:** research and evaluation on I2RT YAM bimanual arms (2× 6-dof arms + |
| grippers, three cameras) for the trained task families: block manipulation, box packing, |
| cable charging. |
| - **Out of scope:** any other embodiment or rig without fine-tuning (VLA policies do not |
| zero-shot transfer across embodiments); unattended operation; operation near people without |
| a hardware e-stop and enforced workspace/torque limits. |
| - **Validation status:** open-loop MSE on one held-out recording session (see Losses & |
| evaluation). No closed-loop sim or real-robot success rates reported. Users are |
| responsible for safe integration (guardrails, e-stop, workspace limits) before deployment. |
|
|
| ## Training |
|
|
| | | | |
| |---|---| |
| | Data | 124 train repos / 5,145 episodes / 11.35M frames; no filtering (full dataset release); held-out val: the separate repo [`allenai/19012026-block-13`](https://huggingface.co/datasets/allenai/19012026-block-13) (session-level holdout) | |
| | Embodiment | YAM bimanual: 14-dof state/action, 2× (6 joints + gripper), absolute joint positions; cameras top + left/right wrists, 360×640 | |
| | Image preprocessing | openpi `yam_pi05` config defaults (see training repo); per-repo loading with 5 ms decode tolerance | |
| | Schedule | 20,000 steps @ batch 512, cosine LR (peak 2.5e-5, warmup 1k), EMA 0.99, 8× B200 (FSDP), ~17.5 h on Modal | |
| | Checkpoint selection | best held-out val over milestone checkpoints every 1,000 steps → step 10,000 | |
| | Headline curve | open-loop val MSE at selection: 0.00206 (reference release on the same protocol: 0.00259 released / 0.00204 best) | |
|
|
| ## Losses & evaluation |
|
|
| - **Training loss:** π0.5's flow-matching action objective (action expert predicts the |
| velocity field for noised 16-step action chunks conditioned on VLM features + state). |
| Full fine-tune (nothing frozen); EMA 0.99 weights are the published ones. |
| - **Eval regime:** open-loop MSE, predicted vs ground-truth actions over 16-step chunks on |
| the held-out session repo, n=20 trajectories, quantile-normalized action space; protocol |
| and script in the [training repo](https://github.com/robocurve/pi05-yam-replication) |
| (`eval_mse.py`). Normalization parity: quantile (q01/q99) stats vendored from the |
| reference release and used identically at train and eval time. |
| - **Comparison:** the reference fine-tune's released checkpoint scores 0.00259 (best 0.00204) |
| on this same protocol; see the training repo for the reference details. |
| - **Scope:** open-loop MSE on a single near-distribution session (an action-prediction |
| proxy), not a closed-loop success rate or task-generalization benchmark. |
|
|
| ## Provenance |
|
|
| | | | |
| |---|---| |
| | Trained by | aris @ [Robocurve](https://huggingface.co/robocurve), 2026-07-02 → 04 | |
| | Training code | https://github.com/robocurve/pi05-yam-replication (private): openpi patches, data prep, train/eval scripts, norm stats | |
| | Framework | [openpi](https://github.com/Physical-Intelligence/openpi) @ `15a9616a00943ada6c20a0f158e3adb39df2ccac` (JAX/Orbax, FSDP) | |
| | Compute provider | [Modal](https://modal.com), 8× NVIDIA B200 (FSDP) | |
| | Wall-clock | ~17.5 h; preemptions: unknown (auto-resume was configured) | |
| | Total compute | ≈140 B200 GPU-hours (8 × 17.5 h); FLOPs not estimated | |
| | Cost | not estimated | |
| | Experiment tracking | wandb (project per training repo config); run not publicly linked | |
| | Card authorship | original card by aris (first-hand); metadata + safety/losses/provenance sections added by the team's publishing session, reconstructed from the training repo and the original card | |
|
|
| ## Format |
|
|
| Orbax checkpoint (JAX): |
| - `params/`: Orbax PyTree of model parameters (EMA weights, ~12 GB) |
| - `assets/yam-bimanual-merged/norm_stats.json`: normalization stats used at train time |
|
|
| ## Usage |
|
|
| With [openpi](https://github.com/Physical-Intelligence/openpi) patched per the |
| [training repo](https://github.com/robocurve/pi05-yam-replication) (registers the |
| `yam_pi05` config): |
|
|
| ```python |
| from openpi.training import config as oc |
| from openpi.policies import policy_config |
| |
| train_cfg = oc.get_config("yam_pi05") |
| policy = policy_config.create_trained_policy(train_cfg, "<local_dir>") |
| action_chunk = policy.infer({ |
| "images": {"top": ..., "left": ..., "right": ...}, # HWC uint8 |
| "state": ..., # (14,) absolute joint positions |
| "prompt": "stack the blocks", |
| })["actions"] # (16, 14) |
| ``` |
|
|
| - **Observations:** cameras `top` → base view, `left`/`right` → wrist views (HWC uint8, |
| 360×640 source); state: (14,) absolute joint positions, 2× (6 joints + gripper). |
| - **Actions:** (16, 14) chunks of absolute joint positions at 30 fps; normalization stats in |
| `assets/yam-bimanual-merged/norm_stats.json` (applied by the policy wrapper). |
|
|
| ## Data provenance & caveats |
|
|
| - All source repos are public AllenAI LeRobot v3 datasets (Apache-2.0); the fine-tune |
| inherits Gemma terms via the PaliGemma backbone (see frontmatter license). |
| - The videos carry small timestamp jitter and occasional frame deficits vs their metadata; |
| training used a 5 ms decode tolerance and per-repo loading (no physical merge; see the |
| training repo README for why merged videos drift). |
| - Val split is a single held-out recording session of the block task: a near-distribution |
| check, not a task-generalization benchmark. No closed-loop or real-robot results. |
|
|
| ## Versioning & contact |
|
|
| - `main` is stable (single published checkpoint, step 10,000). Pin the revision hash for |
| exact reproduction. Superseding checkpoints will set `new_version` here. |
| - Issues: [HF Discussions on this repo](https://huggingface.co/robocurve/pi05-yam-molmoact2/discussions). |
|
|
