Instructions to use robocurve/pi05-yam-molmoact2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use robocurve/pi05-yam-molmoact2 with LeRobot:
- Notebooks
- Google Colab
- Kaggle
docs: remove AI-writing tells from model card (style-only; frontmatter untouched)
Browse files
README.md
CHANGED
|
@@ -31,7 +31,7 @@ model-index:
|
|
| 31 |
value: 0.00206
|
| 32 |
---
|
| 33 |
|
| 34 |
-
# Ο0.5
|
| 35 |
|
| 36 |
[](https://github.com/robocurve/inspect-robots)
|
| 37 |
[](https://github.com/robocurve/inspect-robots-yam)
|
|
@@ -51,38 +51,38 @@ fine-tune on the same data; full recipe at
|
|
| 51 |
- **Intended use:** research and evaluation on I2RT YAM bimanual arms (2Γ 6-dof arms +
|
| 52 |
grippers, three cameras) for the trained task families: block manipulation, box packing,
|
| 53 |
cable charging.
|
| 54 |
-
- **Out of scope:** any other embodiment or rig without fine-tuning
|
| 55 |
-
zero-shot transfer across embodiments; unattended operation; operation near people without
|
| 56 |
a hardware e-stop and enforced workspace/torque limits.
|
| 57 |
- **Validation status:** open-loop MSE on one held-out recording session (see Losses &
|
| 58 |
-
evaluation)
|
| 59 |
responsible for safe integration (guardrails, e-stop, workspace limits) before deployment.
|
| 60 |
|
| 61 |
## Training
|
| 62 |
|
| 63 |
| | |
|
| 64 |
|---|---|
|
| 65 |
-
| Data | 124 train repos / 5,145 episodes / 11.35M frames
|
| 66 |
-
| Embodiment | YAM bimanual: 14-dof state/action
|
| 67 |
| Image preprocessing | openpi `yam_pi05` config defaults (see training repo); per-repo loading with 5 ms decode tolerance |
|
| 68 |
| Schedule | 20,000 steps @ batch 512, cosine LR (peak 2.5e-5, warmup 1k), EMA 0.99, 8Γ B200 (FSDP), ~17.5 h on Modal |
|
| 69 |
-
| Checkpoint selection | best held-out val over milestone checkpoints every 1,000 steps β
|
| 70 |
-
| Headline curve | open-loop val MSE at selection:
|
| 71 |
|
| 72 |
## Losses & evaluation
|
| 73 |
|
| 74 |
- **Training loss:** Ο0.5's flow-matching action objective (action expert predicts the
|
| 75 |
velocity field for noised 16-step action chunks conditioned on VLM features + state).
|
| 76 |
-
Full fine-tune
|
| 77 |
-
- **Eval regime: open-loop MSE
|
| 78 |
the held-out session repo, n=20 trajectories, quantile-normalized action space; protocol
|
| 79 |
and script in the [training repo](https://github.com/robocurve/pi05-yam-replication)
|
| 80 |
(`eval_mse.py`). Normalization parity: quantile (q01/q99) stats vendored from the
|
| 81 |
reference release and used identically at train and eval time.
|
| 82 |
- **Comparison:** the reference fine-tune's released checkpoint scores 0.00259 (best 0.00204)
|
| 83 |
on this same protocol; see the training repo for the reference details.
|
| 84 |
-
- **Scope:** open-loop MSE on a single near-distribution session
|
| 85 |
-
proxy, not a closed-loop success rate or task-generalization benchmark.
|
| 86 |
|
| 87 |
## Provenance
|
| 88 |
|
|
@@ -101,8 +101,8 @@ fine-tune on the same data; full recipe at
|
|
| 101 |
## Format
|
| 102 |
|
| 103 |
Orbax checkpoint (JAX):
|
| 104 |
-
- `params/`
|
| 105 |
-
- `assets/yam-bimanual-merged/norm_stats.json`
|
| 106 |
|
| 107 |
## Usage
|
| 108 |
|
|
@@ -133,9 +133,9 @@ action_chunk = policy.infer({
|
|
| 133 |
- All source repos are public AllenAI LeRobot v3 datasets (Apache-2.0); the fine-tune
|
| 134 |
inherits Gemma terms via the PaliGemma backbone (see frontmatter license).
|
| 135 |
- The videos carry small timestamp jitter and occasional frame deficits vs their metadata;
|
| 136 |
-
training used a 5 ms decode tolerance and per-repo loading (no physical merge
|
| 137 |
training repo README for why merged videos drift).
|
| 138 |
-
- Val split is a single held-out recording session of the block task
|
| 139 |
check, not a task-generalization benchmark. No closed-loop or real-robot results.
|
| 140 |
|
| 141 |
## Versioning & contact
|
|
|
|
| 31 |
value: 0.00206
|
| 32 |
---
|
| 33 |
|
| 34 |
+
# Ο0.5: YAM bimanual fine-tune on MolmoAct2 data
|
| 35 |
|
| 36 |
[](https://github.com/robocurve/inspect-robots)
|
| 37 |
[](https://github.com/robocurve/inspect-robots-yam)
|
|
|
|
| 51 |
- **Intended use:** research and evaluation on I2RT YAM bimanual arms (2Γ 6-dof arms +
|
| 52 |
grippers, three cameras) for the trained task families: block manipulation, box packing,
|
| 53 |
cable charging.
|
| 54 |
+
- **Out of scope:** any other embodiment or rig without fine-tuning (VLA policies do not
|
| 55 |
+
zero-shot transfer across embodiments); unattended operation; operation near people without
|
| 56 |
a hardware e-stop and enforced workspace/torque limits.
|
| 57 |
- **Validation status:** open-loop MSE on one held-out recording session (see Losses &
|
| 58 |
+
evaluation). No closed-loop sim or real-robot success rates reported. Users are
|
| 59 |
responsible for safe integration (guardrails, e-stop, workspace limits) before deployment.
|
| 60 |
|
| 61 |
## Training
|
| 62 |
|
| 63 |
| | |
|
| 64 |
|---|---|
|
| 65 |
+
| Data | 124 train repos / 5,145 episodes / 11.35M frames; no filtering (full dataset release); held-out val: the separate repo [`allenai/19012026-block-13`](https://huggingface.co/datasets/allenai/19012026-block-13) (session-level holdout) |
|
| 66 |
+
| Embodiment | YAM bimanual: 14-dof state/action, 2Γ (6 joints + gripper), absolute joint positions; cameras top + left/right wrists, 360Γ640 |
|
| 67 |
| Image preprocessing | openpi `yam_pi05` config defaults (see training repo); per-repo loading with 5 ms decode tolerance |
|
| 68 |
| Schedule | 20,000 steps @ batch 512, cosine LR (peak 2.5e-5, warmup 1k), EMA 0.99, 8Γ B200 (FSDP), ~17.5 h on Modal |
|
| 69 |
+
| Checkpoint selection | best held-out val over milestone checkpoints every 1,000 steps β step 10,000 |
|
| 70 |
+
| Headline curve | open-loop val MSE at selection: 0.00206 (reference release on the same protocol: 0.00259 released / 0.00204 best) |
|
| 71 |
|
| 72 |
## Losses & evaluation
|
| 73 |
|
| 74 |
- **Training loss:** Ο0.5's flow-matching action objective (action expert predicts the
|
| 75 |
velocity field for noised 16-step action chunks conditioned on VLM features + state).
|
| 76 |
+
Full fine-tune (nothing frozen); EMA 0.99 weights are the published ones.
|
| 77 |
+
- **Eval regime:** open-loop MSE, predicted vs ground-truth actions over 16-step chunks on
|
| 78 |
the held-out session repo, n=20 trajectories, quantile-normalized action space; protocol
|
| 79 |
and script in the [training repo](https://github.com/robocurve/pi05-yam-replication)
|
| 80 |
(`eval_mse.py`). Normalization parity: quantile (q01/q99) stats vendored from the
|
| 81 |
reference release and used identically at train and eval time.
|
| 82 |
- **Comparison:** the reference fine-tune's released checkpoint scores 0.00259 (best 0.00204)
|
| 83 |
on this same protocol; see the training repo for the reference details.
|
| 84 |
+
- **Scope:** open-loop MSE on a single near-distribution session (an action-prediction
|
| 85 |
+
proxy), not a closed-loop success rate or task-generalization benchmark.
|
| 86 |
|
| 87 |
## Provenance
|
| 88 |
|
|
|
|
| 101 |
## Format
|
| 102 |
|
| 103 |
Orbax checkpoint (JAX):
|
| 104 |
+
- `params/`: Orbax PyTree of model parameters (EMA weights, ~12 GB)
|
| 105 |
+
- `assets/yam-bimanual-merged/norm_stats.json`: normalization stats used at train time
|
| 106 |
|
| 107 |
## Usage
|
| 108 |
|
|
|
|
| 133 |
- All source repos are public AllenAI LeRobot v3 datasets (Apache-2.0); the fine-tune
|
| 134 |
inherits Gemma terms via the PaliGemma backbone (see frontmatter license).
|
| 135 |
- The videos carry small timestamp jitter and occasional frame deficits vs their metadata;
|
| 136 |
+
training used a 5 ms decode tolerance and per-repo loading (no physical merge; see the
|
| 137 |
training repo README for why merged videos drift).
|
| 138 |
+
- Val split is a single held-out recording session of the block task: a near-distribution
|
| 139 |
check, not a task-generalization benchmark. No closed-loop or real-robot results.
|
| 140 |
|
| 141 |
## Versioning & contact
|