---
license: apache-2.0
base_model: Qwen/Qwen3-VL-2B-Instruct
tags:
- robotics
- starvla
- vla
- unitree-g1
- inspire-hand
- manipulation
datasets:
- birbirll/g1-inspire-piston-pick-place
---

# StarVLA-OFT — G1 + Inspire piston pick-and-place

Full fine-tune of [StarVLA](https://github.com/starVLA/starVLA) (QwenOFT
framework: Qwen3-VL-2B backbone + MLP L1-regression action head) on
[birbirll/g1-inspire-piston-pick-place](https://huggingface.co/datasets/birbirll/g1-inspire-piston-pick-place)
(102 success episodes, LeRobot v2.1). Vision + language only — **no state
input** (see recipe notes). Closed-loop stable in IsaacLab.

GR00T-N1.6 baseline trained on the same dataset:
[birbirll/g1-inspire-piston-n16](https://huggingface.co/birbirll/g1-inspire-piston-n16).

## Contents

- `final_model/pytorch_model.pt` — bf16 weights (whole model, self-contained)
- `config.yaml` / `config.full.yaml` / `dataset_statistics.json` — the run
  layout StarVLA's `policy_wrapper` expects (config + stats one level above
  the weights)
- `starvla_files/` — drop-ins for your StarVLA checkout:
  - `data_config.py` → `.../train_files/data_registry/` (registers the
    `unitree_g1_piston` embodiment + `unitree_g1_piston_pnp` mixture this
    checkpoint references)
  - `starvla_qwenoft_g1_piston_v2.yaml` — the exact training config
  - `policy_norm_processor_PATCHED.py` — **required to serve this checkpoint**:
    upstream's `PolicyNormProcessor` crashes (`Video key ego_view not found`)
    on any data config whose transform includes video augmentation, as this
    one does. Replace `deployment/model_server/policy_norm_processor.py`.

Point `framework.qwenvl.base_vlm` in `config.yaml` at your local
`Qwen/Qwen3-VL-2B-Instruct` (tokenizer/processor are loaded from there;
the fine-tuned weights come from this repo's `.pt`).

## Recipe (single RTX 4090, ~2h15m; transfers to bigger GPUs)

Action horizon 30, `action_mode: abs`, image 224, color jitter
(0.3/0.4/0.5/0.08), effective batch 32 (micro-batch 8 × grad-accum 4),
10k steps, per-module LRs (backbone 2e-5 / VL interface 1e-5 / head 1e-4),
bnb `PagedAdamW8bit`, gradient checkpointing, **no DeepSpeed** (single GPU).

Three hard-won gotchas if you retrain:

1. **`include_state: false` on purpose.** With state enabled on a stationary
   task, the model regresses actions from proprioception and ignores the
   camera (looks converged, fails closed-loop). StarVLA has no state-dropout
   to mitigate this — leave state off, or wire the dormant
   `StateActionDropout` transform yourself.
2. **Never use `gradient_accumulation_steps > 1` with DeepSpeed** in
   `train_starvla.py` — the trainer steps the LR scheduler per micro-batch,
   silently compressing your schedule. On a single GPU, drop DeepSpeed
   entirely (ZeRO shards nothing across one rank) and accumulation via
   `ACCELERATE_GRADIENT_ACCUMULATION_STEPS` works correctly.
3. **Full fine-tune, don't freeze the VLM.** QwenOFT has no projector —
   freezing the backbone leaves a 42M head learning manipulation from
   web-pretrained features that never adapt. We verified a frozen-VLM sibling
   run converges onto the state shortcut.

## Notes

Works with StarVLA's own `deployment/model_server/server_policy.py`
(WebSocket, msgpack-numpy): request
`{"examples": [{"image": [rgb_hwc_uint8], "lang": "pick up the piston."}],
"unnorm_key": "new_embodiment"}` → `{"data": {"actions": [B, 30, 30]}}`,
absolute physical units.