birbirll/g1-inspire-piston-n16

GR00T N1.6 — G1 + Inspire piston pick-and-place

Fine-tune of nvidia/GR00T-N1.6-3B on birbirll/g1-inspire-piston-pick-place (102 success episodes, LeRobot v2.1). Picks up a piston from a table with the right Inspire hand (Unitree G1, fixed base). Closed-loop verified in IsaacLab.

Contents

• Model weights (bf16 safetensors) + processor/experiment configs — loadable with Isaac-GR00T's run_gr00t_server.py (--embodiment_tag NEW_EMBODIMENT).
• g1_inspire_modality_config.py — the modality config used for training (register via --modality-config-path): state = arms 14 + hands 12 + waist 3 (dims 29:63 of the raw state are unused tactile); action = 30-D (left_arm 7 | right_arm 7 | left_hand 6 | right_hand 6 | base_height 1 | navigate 3), 30-step horizon; arms trained RELATIVE (decoded to absolute by the server).

Training recipe (single RTX 4090, ~2.5 h)

python gr00t/experiment/launch_finetune.py \
  --base-model-path nvidia/GR00T-N1.6-3B \
  --dataset-path <local dataset> \
  --embodiment-tag NEW_EMBODIMENT \
  --modality-config-path g1_inspire_modality_config.py \
  --num-gpus 1 --output-dir ./out \
  --max-steps 10000 --save-steps 1000 --save-total-limit 2 \
  --global-batch-size 8 --gradient-accumulation-steps 4 \
  --state-dropout-prob 0.8 \
  --color-jitter-params brightness 0.3 contrast 0.4 saturation 0.5 hue 0.08

Key knobs: --state-dropout-prob 0.8 (forces vision conditioning — without it the policy shortcuts through proprioception and ignores the camera); effective batch 32; default LR 1e-4; loss ~1.16 → ~0.011. On 24 GB GPUs you must additionally set the optimizer to paged_adamw_8bit and enable gradient checkpointing in launch_finetune.py (upstream defaults OOM); on A100-class hardware the upstream defaults are fine.