birbirll/g1-inspire-piston-pi05

pi0.5 (openpi) — G1 + Inspire piston pick

LoRA fine-tune of pi0.5 (Physical Intelligence openpi, pi05_base) on birbirll/g1-inspire-piston-pick-place (102 success episodes, LeRobot v2.1). Picks up a piston from a table with the right Inspire hand (Unitree G1, fixed base) in IsaacLab. 10 / 10 closed-loop successes — see Eval.

This is the pi05_g1_inspire_piston_lora_abs_st config (v4: absolute actions + discrete state tokens + token-omission dropout), final checkpoint (step 14999).

Siblings on the same dataset: GR00T-N1.6 · StarVLA-OFT.

Base weights (pi05_base) live on GCS (gs://openpi-assets/checkpoints/pi05_base), not HF, so there is no base_model link above — the LoRA adapters here are merged into a full param tree (see Contents).

Contents

Everything an openpi create_trained_policy(...) needs to serve — inference only (optimizer / train_state deliberately excluded):

• params/ — the trained model params (orbax OCDBT PyTree, ~5.9 GB). LoRA adapters are baked into the full param tree; no separate merge step.
• assets/g1-inspire-piston-pick-place-success/norm_stats.json — normalization stats (state + action mean/std/q01/q99). Loaded automatically from the checkpoint at serve time.
• _CHECKPOINT_METADATA — orbax item-handler metadata.
• openpi_files/ — drop-ins for your openpi checkout (see openpi_files/INSTALL.md):
  • config_entry.py — the pi05_g1_inspire_piston_lora_abs_st TrainConfig entry + the LeRobotG1InspireDataConfig class + the two DataConfig dropout fields it references.
  • g1_inspire_policy.py — the G1InspireInputs/G1InspireOutputs transforms (key remap, 29-D proprio slice, 30-D action truncation) → src/openpi/policies/.
  • zmq_adapter_pi05.py — in-process pi0.5 → GR00T-WBC-Bridge ZMQ serving adapter. Usage: python scripts/zmq_adapter_pi05.py --config pi05_g1_inspire_piston_lora_abs_st --ckpt_dir <download_dir> --port 5555 (serves the GR00T PolicyServer REP wire so the existing piston bridge drives this checkpoint with zero bridge changes).
  • INSTALL.md — where each file goes + our openpi fork state vs upstream.

Recipe (verified from the config)

LoRA fine-tune from pi05_base, single RTX 4090.

knob	value
base	pi0.5 (pi05=True), weight_loader = pi05_base/params
LoRA — PaliGemma backbone	gemma_2b_lora: rank 16, alpha 16 (attn + ffn)
LoRA — action expert	gemma_300m_lora: rank 32, alpha 32 (attn + ffn)
steps	15 000 (this ckpt = final, step 14999)
batch size	8
action horizon	10
actions	absolute (use_delta_actions=False) — no DeltaActions at train, no AbsoluteActions re-anchoring at serve
state input	discrete_state_input=True — 29-D proprio discretized to 256 bins, injected into the prompt ("Task: ..., State: <ints>;\nAction: ")
state dropout	state_token_dropout_prob=0.8 — token-omission dropout: 80 % of train samples build the prompt with no state segment (state=None format), NOT zero-fill. Train-only; serving always tokenizes the real state.
norm	quantile (q01/q99), use_quantile_norm=True for pi0.5
EMA	off (LoRA)

Why absolute + token-dropout: earlier state-blind delta variants replayed memorized delta chunks that serving anchored to the drifted live state (correction gain ~0). Absolute actions make drift self-correcting; discrete_state_input=True lets the model see finger closure (ego images are ambiguous about grip), while 80 % token-omission stops the arm trajectory from shortcutting through proprioception.

Layout (verified from the config / modality)

Action — 30-D (G1InspireOutputs truncates the model's 32-D slot to 30):

dims	group
0:7	left_arm (7)
7:14	right_arm (7)
14:20	left_hand (6)
20:26	right_hand (6)
26:27	base_height (1)
27:30	navigate (3)

State — 29-D proprio (the dataset's raw observation.state is 63-D = 29 proprio + 34 tactile; tactile 29:63 is dropped, proprio padded to the model's 32-D slot):

dims	group
0:7	left_arm (7)
7:14	right_arm (7)
14:20	left_hand (6)
20:26	right_hand (6)
26:29	waist (3)

Camera: single ego_view, native 240 × 424 RGB → resized to 224 × 224 by openpi's ModelTransformFactory (ResizeImages(224, 224)). No wrist cameras.

Serving — READ THIS (5 Hz + ACT temporal ensembling REQUIRED)

This checkpoint requires serving at 5 Hz inference with ACT temporal ensembling (canonical Zhao 2023, oldest-chunk-weighted). At its native 2.5 Hz / no ensemble the policy reaches the piston but never commits the grasp — it hovers. The ensembling is what closes the grip.

Bridge flags used for the 10/10 eval:

--inference_hz 5 --temporal_ensemble --ensemble_mode act
# action_hz 25 (wall) inside a 0.5x-realtime sim = dataset-native 50 Hz sim-time

Also set XLA_PYTHON_CLIENT_PREALLOCATE=false when the policy is co-resident with the sim on a single GPU (otherwise JAX preallocates and starves the sim).

Start the policy server (from an openpi checkout with its .venv, after installing the openpi_files/ drop-ins):

# ckpt_dir = the directory you downloaded this repo into (contains params/, assets/, _CHECKPOINT_METADATA)
XLA_PYTHON_CLIENT_PREALLOCATE=false .venv/bin/python scripts/zmq_adapter_pi05.py \
  --config  pi05_g1_inspire_piston_lora_abs_st \
  --ckpt_dir <download_dir> \
  --port 5555

The adapter loads the openpi Policy in-process and speaks the GR00T PolicyServer ZMQ REP wire, so the existing piston bridge (--server_codec custom --transport zmq_isaac --server_port 5555) drives it unchanged. policy.infer(...) returns (horizon=10, 30) absolute physical-unit joint targets, sliced by the fixed column plan above.

Eval

Closed-loop in IsaacLab (piston pick, right Inspire hand, G1 fixed base):

• 10 / 10 success over the counted closed-loop episodes.
• Mean sustained lift 4.1 s; all 10 episodes reached the 4.0 s sustain ceiling (the eval's success/hold cutoff).
• Caveat: fixed object pose — no pose randomization in these episodes. Generalization to randomized piston placement is unverified here.

License

Apache-2.0 (inherits openpi).