--- license: apache-2.0 tags: - robotics - openpi - pi0 - pi05 - lora - unitree-g1 - inspire-hand - manipulation datasets: - birbirll/g1-inspire-piston-pick-place --- # pi0.5 (openpi) — G1 + Inspire piston pick LoRA fine-tune of **pi0.5** (Physical Intelligence `openpi`, `pi05_base`) on [birbirll/g1-inspire-piston-pick-place](https://huggingface.co/datasets/birbirll/g1-inspire-piston-pick-place) (102 success episodes, LeRobot v2.1). Picks up a piston from a table with the right Inspire hand (Unitree G1, fixed base) in IsaacLab. **10 / 10 closed-loop successes** — see [Eval](#eval). This is the `pi05_g1_inspire_piston_lora_abs_st` config (v4: absolute actions + discrete state tokens + token-omission dropout), final checkpoint (step 14999). Siblings on the same dataset: [GR00T-N1.6](https://huggingface.co/birbirll/g1-inspire-piston-n16) · [StarVLA-OFT](https://huggingface.co/birbirll/g1-inspire-piston-starvla-oft). > Base weights (`pi05_base`) live on GCS (`gs://openpi-assets/checkpoints/pi05_base`), > not HF, so there is no `base_model` link above — the LoRA adapters here are > merged into a full param tree (see [Contents](#contents)). ## Contents Everything an openpi `create_trained_policy(...)` needs to serve — **inference only** (optimizer / train_state deliberately excluded): - `params/` — the trained model params (orbax OCDBT PyTree, ~5.9 GB). LoRA adapters are baked into the full param tree; no separate merge step. - `assets/g1-inspire-piston-pick-place-success/norm_stats.json` — normalization stats (state + action mean/std/q01/q99). Loaded automatically from the checkpoint at serve time. - `_CHECKPOINT_METADATA` — orbax item-handler metadata. - `openpi_files/` — drop-ins for your openpi checkout (see `openpi_files/INSTALL.md`): - `config_entry.py` — the `pi05_g1_inspire_piston_lora_abs_st` `TrainConfig` entry + the `LeRobotG1InspireDataConfig` class + the two `DataConfig` dropout fields it references. - `g1_inspire_policy.py` — the `G1InspireInputs`/`G1InspireOutputs` transforms (key remap, 29-D proprio slice, 30-D action truncation) → `src/openpi/policies/`. - `zmq_adapter_pi05.py` — in-process pi0.5 → GR00T-WBC-Bridge ZMQ serving adapter. **Usage:** `python scripts/zmq_adapter_pi05.py --config pi05_g1_inspire_piston_lora_abs_st --ckpt_dir --port 5555` (serves the GR00T `PolicyServer` REP wire so the existing piston bridge drives this checkpoint with zero bridge changes). - `INSTALL.md` — where each file goes + our openpi fork state vs upstream. ## Recipe (verified from the config) LoRA fine-tune from `pi05_base`, single RTX 4090. | knob | value | |---|---| | base | pi0.5 (`pi05=True`), `weight_loader = pi05_base/params` | | LoRA — PaliGemma backbone | `gemma_2b_lora`: rank **16**, alpha 16 (attn + ffn) | | LoRA — action expert | `gemma_300m_lora`: rank **32**, alpha 32 (attn + ffn) | | steps | **15 000** (this ckpt = final, step **14999**) | | batch size | 8 | | action horizon | 10 | | actions | **absolute** (`use_delta_actions=False`) — no DeltaActions at train, no AbsoluteActions re-anchoring at serve | | state input | `discrete_state_input=True` — 29-D proprio discretized to 256 bins, injected into the prompt (`"Task: ..., State: ;\nAction: "`) | | state dropout | `state_token_dropout_prob=0.8` — **token-omission** dropout: 80 % of train samples build the prompt with **no** state segment (state=None format), NOT zero-fill. Train-only; serving always tokenizes the real state. | | norm | quantile (q01/q99), `use_quantile_norm=True` for pi0.5 | | EMA | off (LoRA) | Why absolute + token-dropout: earlier state-blind delta variants replayed memorized delta chunks that serving anchored to the drifted live state (correction gain ~0). Absolute actions make drift self-correcting; `discrete_state_input=True` lets the model see finger closure (ego images are ambiguous about grip), while 80 % token-omission stops the arm trajectory from shortcutting through proprioception. ## Layout (verified from the config / modality) **Action — 30-D** (`G1InspireOutputs` truncates the model's 32-D slot to 30): | dims | group | |---|---| | 0:7 | left_arm (7) | | 7:14 | right_arm (7) | | 14:20 | left_hand (6) | | 20:26 | right_hand (6) | | 26:27 | base_height (1) | | 27:30 | navigate (3) | **State — 29-D** proprio (the dataset's raw `observation.state` is 63-D = 29 proprio + 34 tactile; tactile 29:63 is dropped, proprio padded to the model's 32-D slot): | dims | group | |---|---| | 0:7 | left_arm (7) | | 7:14 | right_arm (7) | | 14:20 | left_hand (6) | | 20:26 | right_hand (6) | | 26:29 | waist (3) | **Camera:** single `ego_view`, native **240 × 424** RGB → resized to **224 × 224** by openpi's `ModelTransformFactory` (`ResizeImages(224, 224)`). No wrist cameras. ## Serving — READ THIS (5 Hz + ACT temporal ensembling REQUIRED) This checkpoint **requires serving at 5 Hz inference with ACT temporal ensembling** (canonical Zhao 2023, oldest-chunk-weighted). At its native **2.5 Hz / no ensemble** the policy **reaches the piston but never commits the grasp — it hovers.** The ensembling is what closes the grip. Bridge flags used for the 10/10 eval: ```bash --inference_hz 5 --temporal_ensemble --ensemble_mode act # action_hz 25 (wall) inside a 0.5x-realtime sim = dataset-native 50 Hz sim-time ``` Also set `XLA_PYTHON_CLIENT_PREALLOCATE=false` when the policy is co-resident with the sim on a single GPU (otherwise JAX preallocates and starves the sim). Start the policy server (from an openpi checkout with its `.venv`, after installing the `openpi_files/` drop-ins): ```bash # ckpt_dir = the directory you downloaded this repo into (contains params/, assets/, _CHECKPOINT_METADATA) XLA_PYTHON_CLIENT_PREALLOCATE=false .venv/bin/python scripts/zmq_adapter_pi05.py \ --config pi05_g1_inspire_piston_lora_abs_st \ --ckpt_dir \ --port 5555 ``` The adapter loads the openpi `Policy` in-process and speaks the GR00T `PolicyServer` ZMQ REP wire, so the existing piston bridge (`--server_codec custom --transport zmq_isaac --server_port 5555`) drives it unchanged. `policy.infer(...)` returns `(horizon=10, 30)` absolute physical-unit joint targets, sliced by the fixed column plan above. ## Eval Closed-loop in IsaacLab (piston pick, right Inspire hand, G1 fixed base): - **10 / 10 success** over the counted closed-loop episodes. - Mean sustained lift **4.1 s**; all 10 episodes reached the **4.0 s sustain ceiling** (the eval's success/hold cutoff). - **Caveat:** fixed object pose — **no pose randomization** in these episodes. Generalization to randomized piston placement is unverified here. ## License Apache-2.0 (inherits openpi).