README.md

---
# library_name omitted: openpi (JAX) has no Hub library integration
# base_model omitted: fine-tuned from openpi's original JAX pi05_base release
#   (gs://openpi-assets), which has no canonical Hub repo; the PyTorch
#   conversion lerobot/pi05_base is a sibling artifact, not this run's base.
pipeline_tag: robotics
license: gemma
datasets:
  - allenai/MolmoAct2-BimanualYAM-Dataset
tags:
  - robotics
  - vla
  - openpi
  - pi05
  - bimanual
  - yam
  - jax
  - lerobot
inference: false
model-index:
  - name: pi05-yam-molmoact2
    results:
      - task:
          type: robotics
        dataset:
          type: allenai/19012026-block-13
          name: held-out YAM episode repo (open-loop)
        metrics:
          - type: open_loop_mse
            name: open-loop action MSE (16-step chunks, n=20)
            value: 0.00206
---

# π0.5: YAM bimanual fine-tune on MolmoAct2 data

[![Evaluate with Inspect Robots](https://img.shields.io/badge/evaluate%20with-Inspect%20Robots-indigo)](https://github.com/robocurve/inspect-robots)
[![Adapters: YAM](https://img.shields.io/badge/adapters-inspect--robots--yam-blue)](https://github.com/robocurve/inspect-robots-yam)
[![Benchmarks: WorldEvals](https://img.shields.io/badge/benchmarks-WorldEvals-2ea44f)](https://worldevals.org)
[![Catalog: WorldPolicies](https://img.shields.io/badge/catalog-WorldPolicies-yellow)](https://huggingface.co/collections/robocurve/worldpolicies-6a4dc8fd556a82aeea0fca37)

JAX/Orbax fine-tune of [openpi](https://github.com/Physical-Intelligence/openpi)
`pi05_base` (full fine-tune, nothing frozen; PaliGemma backbone → Gemma license terms)
for the bimanual YAM platform, trained on the AllenAI
[MolmoAct2-BimanualYAM dataset](https://huggingface.co/datasets/allenai/MolmoAct2-BimanualYAM-Dataset)
(124 LeRobot repos, 5,145 episodes, 11.35M frames). Independent replication of a reference
fine-tune on the same data; full recipe at
[robocurve/pi05-yam-replication](https://github.com/robocurve/pi05-yam-replication).

## Intended use & safety

- **Intended use:** research and evaluation on I2RT YAM bimanual arms (2× 6-dof arms +
  grippers, three cameras) for the trained task families: block manipulation, box packing,
  cable charging.
- **Out of scope:** any other embodiment or rig without fine-tuning (VLA policies do not
  zero-shot transfer across embodiments); unattended operation; operation near people without
  a hardware e-stop and enforced workspace/torque limits.
- **Validation status:** open-loop MSE on one held-out recording session (see Losses &
  evaluation). No closed-loop sim or real-robot success rates reported. Users are
  responsible for safe integration (guardrails, e-stop, workspace limits) before deployment.

## Training

| | |
|---|---|
| Data | 124 train repos / 5,145 episodes / 11.35M frames; no filtering (full dataset release); held-out val: the separate repo [`allenai/19012026-block-13`](https://huggingface.co/datasets/allenai/19012026-block-13) (session-level holdout) |
| Embodiment | YAM bimanual: 14-dof state/action, 2× (6 joints + gripper), absolute joint positions; cameras top + left/right wrists, 360×640 |
| Image preprocessing | openpi `yam_pi05` config defaults (see training repo); per-repo loading with 5 ms decode tolerance |
| Schedule | 20,000 steps @ batch 512, cosine LR (peak 2.5e-5, warmup 1k), EMA 0.99, 8× B200 (FSDP), ~17.5 h on Modal |
| Checkpoint selection | best held-out val over milestone checkpoints every 1,000 steps → step 10,000 |
| Headline curve | open-loop val MSE at selection: 0.00206 (reference release on the same protocol: 0.00259 released / 0.00204 best) |

## Losses & evaluation

- **Training loss:** π0.5's flow-matching action objective (action expert predicts the
  velocity field for noised 16-step action chunks conditioned on VLM features + state).
  Full fine-tune (nothing frozen); EMA 0.99 weights are the published ones.
- **Eval regime:** open-loop MSE, predicted vs ground-truth actions over 16-step chunks on
  the held-out session repo, n=20 trajectories, quantile-normalized action space; protocol
  and script in the [training repo](https://github.com/robocurve/pi05-yam-replication)
  (`eval_mse.py`). Normalization parity: quantile (q01/q99) stats vendored from the
  reference release and used identically at train and eval time.
- **Comparison:** the reference fine-tune's released checkpoint scores 0.00259 (best 0.00204)
  on this same protocol; see the training repo for the reference details.
- **Scope:** open-loop MSE on a single near-distribution session (an action-prediction
  proxy), not a closed-loop success rate or task-generalization benchmark.

## Provenance

| | |
|---|---|
| Trained by | aris @ [Robocurve](https://huggingface.co/robocurve), 2026-07-02 → 04 |
| Training code | https://github.com/robocurve/pi05-yam-replication (private): openpi patches, data prep, train/eval scripts, norm stats |
| Framework | [openpi](https://github.com/Physical-Intelligence/openpi) @ `15a9616a00943ada6c20a0f158e3adb39df2ccac` (JAX/Orbax, FSDP) |
| Compute provider | [Modal](https://modal.com), 8× NVIDIA B200 (FSDP) |
| Wall-clock | ~17.5 h; preemptions: unknown (auto-resume was configured) |
| Total compute | ≈140 B200 GPU-hours (8 × 17.5 h); FLOPs not estimated |
| Cost | not estimated |
| Experiment tracking | wandb (project per training repo config); run not publicly linked |
| Card authorship | original card by aris (first-hand); metadata + safety/losses/provenance sections added by the team's publishing session, reconstructed from the training repo and the original card |

## Format

Orbax checkpoint (JAX):
- `params/`: Orbax PyTree of model parameters (EMA weights, ~12 GB)
- `assets/yam-bimanual-merged/norm_stats.json`: normalization stats used at train time

## Usage

With [openpi](https://github.com/Physical-Intelligence/openpi) patched per the
[training repo](https://github.com/robocurve/pi05-yam-replication) (registers the
`yam_pi05` config):

```python
from openpi.training import config as oc
from openpi.policies import policy_config

train_cfg = oc.get_config("yam_pi05")
policy = policy_config.create_trained_policy(train_cfg, "<local_dir>")
action_chunk = policy.infer({
    "images": {"top": ..., "left": ..., "right": ...},  # HWC uint8
    "state": ...,                                        # (14,) absolute joint positions
    "prompt": "stack the blocks",
})["actions"]                                            # (16, 14)
```

- **Observations:** cameras `top` → base view, `left`/`right` → wrist views (HWC uint8,
  360×640 source); state: (14,) absolute joint positions, 2× (6 joints + gripper).
- **Actions:** (16, 14) chunks of absolute joint positions at 30 fps; normalization stats in
  `assets/yam-bimanual-merged/norm_stats.json` (applied by the policy wrapper).

## Data provenance & caveats

- All source repos are public AllenAI LeRobot v3 datasets (Apache-2.0); the fine-tune
  inherits Gemma terms via the PaliGemma backbone (see frontmatter license).
- The videos carry small timestamp jitter and occasional frame deficits vs their metadata;
  training used a 5 ms decode tolerance and per-repo loading (no physical merge; see the
  training repo README for why merged videos drift).
- Val split is a single held-out recording session of the block task: a near-distribution
  check, not a task-generalization benchmark. No closed-loop or real-robot results.

## Versioning & contact

- `main` is stable (single published checkpoint, step 10,000). Pin the revision hash for
  exact reproduction. Superseding checkpoints will set `new_version` here.
- Issues: [HF Discussions on this repo](https://huggingface.co/robocurve/pi05-yam-molmoact2/discussions).