eval: publish closed-loop 12-position grid results + photo (checkpoint-20000)

1caaff4 17 days ago

16.2 kB

license: other
license_name: nvidia-open-model-license-agreement
license_link: >-
  https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
library_name: gr00t
base_model: nvidia/GR00T-N1.7-3B
base_model_relation: finetune
datasets:
  - cloudwalk-research/gr00t-g1-grab-bottle-right-hand-radius-20-merged
tags:
  - gr00t
  - gr00t-n1.7
  - vla
  - vision-language-action
  - humanoid
  - robotics
  - imitation-learning
  - diffusion-policy
  - unitree-g1
  - inspire-ftp
  - sonic-wbc
  - arxiv:2503.14734
  - arxiv:2511.07820
language:
  - en
pipeline_tag: robotics

GR00T N1.7-3B Fine-Tune v6 — Unitree G1 "grab the bottle" (right hand, SONIC WBC, merged radius-20 dataset)

v6 fine-tune of NVIDIA GR00T N1.7-3B (paper) on a Unitree G1 humanoid driven by the SONIC whole-body controller.

Single-handed pick task ("grab the bottle", RIGHT hand). Trained at the CloudWalk Robotics Lab (CW-RL), 2026-06, on a 502-episode / 120,017-frame real-robot teleoperation dataset — the largest and best-curated of the lineage — for the UNITREE_G1_SONIC embodiment. Released as a reference fine-tune for teams building manipulation policies on the GR00T + SONIC + MuJoCo/G1 stack.

Predecessors: v2 (…-210ep-v2-finetune) — the current validated production champion (checkpoint-20000, grasps from poses where v1 failed) — v4 (…-417ep-v4-finetune) (radius-5, 6/12), v5 (…-314ep-v5-finetune) (radius-20, 9/12), and v1 (…-105ep-v1-finetune). v6's dataset is a merge of two grab-bottle sets — the 105-ep set + a new 115-ep "worst-positions" set (hard poses) — re-curated at radius 20 with two filtering improvements over prior versions: (a) frame removal near the grasp moment is now prohibited (a defect that had slipped through and likely hurt earlier versions' grasp behavior), and (b) episodes 95–114 were kept as-is (curation skipped). The result is 482 clean curated segments + 20 as-is = 502 episodes (20.3% of frames / 29.5% of samples removed elsewhere), ~239 frames/ep — 2.4× the frames of v5.

This is a behavior-cloning fine-tune of the full 3B model. Status: trained 2026-06-24, closed-loop eval complete (8/12) — only checkpoint-20000 is published (one checkpoint per repo, lineage storage policy); the larger merged set underfit and did not beat the v2 champion (see Evaluation).

Quick facts


Base	GR00T-N1.7-3B — Qwen3-VL vision-language backbone + flow-matching diffusion-transformer (DiT) action head
Parameters	3.14 B total / 1.62 B trainable (51.5%)
Dataset	cloudwalk-research/gr00t-g1-grab-bottle-right-hand-radius-20-merged — 502 episodes, 120,017 frames @ 50 Hz, 640×480 `ego_view` camera (no wrist cams); merge of the 105-ep + a new 115-ep "worst-positions" set, radius-20 curation with grasp-frame preservation (~239 frames/ep)
Robot target	Unitree G1 (29-DoF body) + Inspire FTP hands (7-DoF/hand: 6 finger joints + 1 grip pad, via InspireFTPGripMapper) + SONIC whole-body controller
Embodiment tag	`UNITREE_G1_SONIC` (`unitree_g1_sonic`)
State space	43-D (left_leg 6 + right_leg 6 + waist 3 + left_arm 7 + left_hand 7 + right_arm 7 + right_hand 7)
Action space	`[40 × 78]` = 40-step horizon × (64 motion_token + 7 left_hand_joints + 7 right_hand_joints)
Hardware	6× NVIDIA B200 (sm_100 / Blackwell)
Mixed precision	bf16
Optimizer	AdamW, lr 1e-4 cosine, warmup_ratio 0.05, weight_decay 1e-5
Steps / batch	20,000 / global batch 48 (8 per GPU × 6 GPUs); only `checkpoint-20000` is published (one checkpoint per repo, lineage storage policy)
Epochs	~8.0 @ 20k (960k frame-views ÷ 120,017 frames); 10k ≈ 4.0
Augmentation	color jitter (brightness 0.3, contrast 0.4, saturation 0.5, hue 0.08)
Wall-clock	~65.3 min (`train_runtime` 3915.8 s · 6× B200 @ 5.108 steps/s, 245 samples/s)
Final train loss	0.0866 (mean over run; per-step @20k ≈ 0.0354)
W&B run	`0p48c1z0` (offline → synced; project `g1_grab_bottle`)

Repository contents

checkpoint-20000/    # 20k steps (~8.0 ep) — the only published checkpoint
README.md            # this file

Each checkpoint-NNNNN/ is a self-contained, deploy-ready snapshot:

model-00001-of-00002.safetensors + model-00002-of-00002.safetensors (~6.5 GB, bf16)
model.safetensors.index.json
config.json
embodiment_id.json (contains unitree_g1_sonic)
processor_config.json
statistics.json (dataset normalization stats)
experiment_cfg/ (training config snapshot)

DeepSpeed ZeRO partition states, optimizer, scheduler and RNG files are intentionally omitted — they are needed only to resume training and would add several GB per checkpoint.

Evaluation

Why checkpoint-20000 is the goal. On the 120,017-frame merged set, 20,000 steps is ~8.0 epochs — below v2's validated sweet spot (~15.3 epochs), because the dataset is 2.4× larger than v5's. The live risk here is therefore underfit, not overfit. Only checkpoint-20000 is published (one checkpoint per repo, lineage storage policy); it is tested on the real robot first. The production checkpoint is chosen by closed-loop comparison, not by training loss. There is no held-out split; train loss is a fit probe, not a generalization measure.

Extension to 40k — no longer a true-resume. A clean continuation from checkpoint-20000 to 40,000 steps would have produced checkpoint-30000 (~~12 ep) and checkpoint-40000 (~~16 ep), bracketing v2's sweet spot. However, the cluster training state (DeepSpeed/optimizer/scheduler) was cleaned from NFS on 2026-06-24, so a true-resume is no longer possible — reaching 40k now requires re-training from scratch (MAX_STEPS=40000). The published deploy weights are unaffected (eval/inference unchanged).

Closed-loop evaluation — completed (real G1). checkpoint-20000 was run closed-loop on the physical G1 + SONIC stack and measured against the v2 checkpoint-20000 champion. Low BC loss is a healthy prerequisite, not evidence of a good policy — this closed-loop result is the load-bearing signal.

Procedure. The bottle was placed at 12 fixed positions spanning one half-side of the table (the table is fixed relative to the robot). These are the same 12 positions used for data collection (see the source dataset card). One trial per position (12 trials) gives the grid result. Legend: 🟥 red = knocked the bottle over · 🟧 orange = stuck in an "indecision" loop, never reaching the bottle · 🟩 green = successfully grasped the bottle.

Results — checkpoint-20000.

Grid (12 fixed positions, 1 trial each):

Outcome	Count	Rate
🟩 Success	8/12	67%
🟧 Non-convergence (indecision loop)	3/12	25%
🟥 Error (knocked bottle over)	1/12	8%

Conclusion. More data did not beat better data. v6 trains on the largest set (502 episodes / 120,017 frames, 2.4× v5) yet lands below both v2 (10/12) and v5 (9/12). Its bright spot is the tied-lowest knock-over rate (1/12, 8%, tied with v2) — the merged set's worst-position coverage taught it to avoid toppling the bottle — but the highest non-convergence (3/12, 25%) shows the 2.4× larger set at only ~8 epochs underfit: the policy often can't commit. The lineage already flagged underfit as the live risk; the closed-loop result confirms it. v2 remains in production.

How to download

from huggingface_hub import snapshot_download

local = snapshot_download(
    repo_id="cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-502ep-v6-finetune",
    repo_type="model",
    allow_patterns=["checkpoint-20000/*"],
)
print(local)

How to deploy

Start the GR00T policy server (from an Isaac-GR00T environment) pointing at the downloaded checkpoint:

python -m gr00t.eval.run_gr00t_server \
    --model-path <local>/checkpoint-20000 \
    --embodiment-tag UNITREE_G1_SONIC \
    --device cuda:0 --host 0.0.0.0 --port 5550

Closed-loop control of the G1 (sim or real) is driven by the SONIC whole-body controller in GR00T-WholeBodyControl: the policy emits motion_token + hand-joint targets that the SONIC WBC decodes into whole-body joint commands. The server must be launched with the same UNITREE_G1_SONIC embodiment tag used in training. See the NVlabs VLA inference tutorial. This checkpoint is not plug-and-play on hardware — it requires the SONIC C++ deploy stack and the matching G1 setup.

Known caveats

Right-hand-only, single task. The dataset is one task ("grab the bottle") executed with the right hand. Left-hand and locomotion action dims reflect the (largely stationary) demonstrations; do not expect bimanual or locomotion behavior.
Single camera. Only the ego_view (head) camera was recorded — no wrist cameras. The policy has never seen wrist views.
Deployment needs the SONIC stack. The checkpoint outputs motion_token + hand joints for UNITREE_G1_SONIC; it only produces robot motion through the SONIC WBC + ZMQ deploy pipeline. It is not directly executable on a bare G1.
Train loss is not held-out. No episode split; the loss is a smoothed train-fit probe, not a generalization metric.
Underfit risk (not overfit). 20,000 steps = ~~8.0 epochs on 120,017 frames — below v2's validated ~15.3-epoch sweet spot, because the dataset is 2.4× larger. The live risk is under-training, not overfitting. checkpoint-20000 (~~8.0 ep) is the goal; reaching 40k (~16 ep) is now a from-scratch re-train (the resume state was cleaned from NFS 2026-06-24) — see Evaluation.
Single published checkpoint. Only checkpoint-20000 is published (one checkpoint per repo, lineage storage policy). There is no intermediate rung — a 40k extension (now only via from-scratch re-train) would add checkpoint-30000 and checkpoint-40000.
Merged dataset provenance. v6's data is a merge of two source sets (the 105-ep set + a new 115-ep "worst-positions" set), re-curated at radius 20 with grasp-frame preservation (frames near the grasp are no longer removed) and episodes 95–114 kept as-is (482 curated segments + 20 as-is = 502). The grasp-frame fix is a behavioral improvement over earlier versions whose effect is to be validated in closed-loop.
Training environment. Trained on a shared B200 node with NCCL_IB_DISABLE=1 NCCL_P2P_LEVEL=NVL (InfiniBand off, P2P over NVLink) and W&B in offline mode (synced post-run). These affect only the training run, not the weights.

Lineage

Version	Dataset	Episodes	Frames	Epochs @20k	Closed-loop	Notes
v1	105ep-v1	105	70,680	13.6	✅ 4/12 (33%)	First GR00T N1.7 + SONIC fine-tune at CW-RL; right-hand bottle pick.
v2	right-hand-v2	210	62,772	15.3	✅ 10/12 (83%) — champion	Curated (episodes split into shorter windows, bad segments removed). `checkpoint-20000` = current production champion (10/12).
v4	radius-5	417	48,577	19.8	✅ 6/12 (50%)	Zero-wandering, most aggressive curation (radius 5 → short windows).
v5	radius-20	314	50,496	19.0	✅ 9/12 (75%)	Zero-wandering, least aggressive curation (radius 20 → longer windows, ~161 f/ep).
v6 (this)	radius-20-merged	502	120,017	8.0	✅ 8/12 (67%)	Merged (105-ep + 115-ep "worst-positions"), radius-20 with grasp-frame preservation. Largest set → 20k = only ~8 ep (underfit risk).
v7	speedup-3mm-v1	220	60,163	~16.0	⏳ TBD	DP speedup resampling (wrist-Cartesian, 3 mm/frame, max_K=40) of the merged set; no segment removal — reference baseline. ≈16 ep @20k lands on v2's sweet spot.

References

GR00T N1 — Open foundation model for generalist humanoid robots; base policy fine-tuned here. Paper (arXiv:2503.14734), base model.
Isaac-GR00T — Training / inference / deployment stack used for this fine-tune. GitHub.
GR00T-WholeBodyControl (SONIC) — Whole-body controller + teleop + VLA deployment for the G1. GitHub, SONIC paper (arXiv:2511.07820).
Training dataset — cloudwalk-research/gr00t-g1-grab-bottle-right-hand-radius-20-merged (CloudWalk Research, 2026), PICO 4 Ultra teleoperation on the Unitree G1 with SONIC WBC; a merge of the 105-ep set + a new 115-ep "worst-positions" set, re-curated at radius 20 with grasp-frame preservation (502 episodes, 120,017 frames).

Attribution

Developed by cloudwalk-research in the CloudWalk Robotics Lab (CW-RL). Fine-tuned from nvidia/GR00T-N1.7-3B using Isaac-GR00T; targets the Unitree G1 with the SONIC whole-body controller.

Citation

@misc{cwrl_gr00t_grab_bottle_v6_2026,
  title        = {GR00T N1.7 Fine-Tune v6 --- Unitree G1 "grab the bottle" (right hand, SONIC WBC, merged radius-20 curated 502-ep dataset)},
  author       = {{CloudWalk Robotics Lab}},
  year         = {2026},
  howpublished = {Hugging Face model repository},
  url          = {https://huggingface.co/cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-502ep-v6-finetune}
}

License

Inherits the NVIDIA Open Model License Agreement of the base model nvidia/GR00T-N1.7-3B — see the license terms. This is a research preview: not intended for safety-critical use; closed-loop deployment on a physical humanoid requires human oversight.