--- license: other license_name: nvidia-open-model-license-agreement license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/ library_name: gr00t base_model: nvidia/GR00T-N1.7-3B base_model_relation: finetune datasets: - cloudwalk-research/gr00t-g1-grab-bottle-right-hand-radius-20-merged tags: - gr00t - gr00t-n1.7 - vla - vision-language-action - humanoid - robotics - imitation-learning - diffusion-policy - unitree-g1 - inspire-ftp - sonic-wbc - arxiv:2503.14734 - arxiv:2511.07820 language: - en pipeline_tag: robotics --- # GR00T N1.7-3B Fine-Tune v6 — Unitree G1 "grab the bottle" (right hand, SONIC WBC, merged radius-20 dataset) **v6 fine-tune of [NVIDIA GR00T N1.7-3B](https://huggingface.co/nvidia/GR00T-N1.7-3B) ([paper](https://arxiv.org/abs/2503.14734)) on a Unitree G1 humanoid driven by the [SONIC](https://arxiv.org/abs/2511.07820) whole-body controller.** Single-handed pick task ("grab the bottle", RIGHT hand). Trained at the **CloudWalk Robotics Lab (CW-RL)**, 2026-06, on a **502-episode / 120,017-frame** real-robot teleoperation dataset — the **largest and best-curated** of the lineage — for the `UNITREE_G1_SONIC` embodiment. Released as a reference fine-tune for teams building manipulation policies on the GR00T + SONIC + MuJoCo/G1 stack. **Predecessors:** [v2 (`…-210ep-v2-finetune`)](https://huggingface.co/cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-210ep-v2-finetune) — the current **validated production champion** (`checkpoint-20000`, grasps from poses where v1 failed) — [v4 (`…-417ep-v4-finetune`)](https://huggingface.co/cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-417ep-v4-finetune) (radius-5, 6/12), [v5 (`…-314ep-v5-finetune`)](https://huggingface.co/cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-314ep-v5-finetune) (radius-20, 9/12), and [v1 (`…-105ep-v1-finetune`)](https://huggingface.co/cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-105ep-v1-finetune). v6's dataset is a **merge of two grab-bottle sets** — the 105-ep set + a new **115-ep "worst-positions" set** (hard poses) — re-curated at **radius 20** with two filtering improvements over prior versions: **(a) frame removal near the grasp moment is now prohibited** (a defect that had slipped through and likely hurt earlier versions' grasp behavior), and **(b) episodes 95–114 were kept as-is** (curation skipped). The result is **482 clean curated segments + 20 as-is = 502 episodes** (20.3% of frames / 29.5% of samples removed elsewhere), ~239 frames/ep — 2.4× the frames of v5. This is a behavior-cloning fine-tune of the full 3B model. **Status: trained 2026-06-24, closed-loop eval complete (8/12)** — only `checkpoint-20000` is published (one checkpoint per repo, lineage storage policy); the larger merged set underfit and did not beat the v2 champion (see [Evaluation](#evaluation)). ## Quick facts | | | | --- | --- | | Base | [GR00T-N1.7-3B](https://huggingface.co/nvidia/GR00T-N1.7-3B) — Qwen3-VL vision-language backbone + flow-matching diffusion-transformer (DiT) action head | | Parameters | 3.14 B total / 1.62 B trainable (51.5%) | | Dataset | [cloudwalk-research/gr00t-g1-grab-bottle-right-hand-radius-20-merged](https://huggingface.co/datasets/cloudwalk-research/gr00t-g1-grab-bottle-right-hand-radius-20-merged) — **502 episodes, 120,017 frames** @ 50 Hz, 640×480 `ego_view` camera (no wrist cams); merge of the 105-ep + a new 115-ep "worst-positions" set, radius-20 curation with grasp-frame preservation (~239 frames/ep) | | Robot target | Unitree G1 (29-DoF body) + Inspire FTP hands (7-DoF/hand: 6 finger joints + 1 grip pad, via InspireFTPGripMapper) + SONIC whole-body controller | | Embodiment tag | `UNITREE_G1_SONIC` (`unitree_g1_sonic`) | | State space | 43-D (left_leg 6 + right_leg 6 + waist 3 + left_arm 7 + left_hand 7 + right_arm 7 + right_hand 7) | | Action space | `[40 × 78]` = 40-step horizon × (64 motion_token + 7 left_hand_joints + 7 right_hand_joints) | | Hardware | 6× NVIDIA B200 (sm_100 / Blackwell) | | Mixed precision | bf16 | | Optimizer | AdamW, lr 1e-4 cosine, warmup_ratio 0.05, weight_decay 1e-5 | | Steps / batch | 20,000 / global batch 48 (8 per GPU × 6 GPUs); only `checkpoint-20000` is published (one checkpoint per repo, lineage storage policy) | | Epochs | ~8.0 @ 20k (960k frame-views ÷ 120,017 frames); 10k ≈ 4.0 | | Augmentation | color jitter (brightness 0.3, contrast 0.4, saturation 0.5, hue 0.08) | | Wall-clock | **~65.3 min** (`train_runtime` 3915.8 s · 6× B200 @ **5.108 steps/s**, 245 samples/s) | | Final train loss | **0.0866** (mean over run; per-step @20k ≈ 0.0354) | | W&B run | `0p48c1z0` (offline → synced; project `g1_grab_bottle`) | ## Repository contents ``` checkpoint-20000/ # 20k steps (~8.0 ep) — the only published checkpoint README.md # this file ``` Each `checkpoint-NNNNN/` is a self-contained, deploy-ready snapshot: - `model-00001-of-00002.safetensors` + `model-00002-of-00002.safetensors` (~6.5 GB, bf16) - `model.safetensors.index.json` - `config.json` - `embodiment_id.json` (contains `unitree_g1_sonic`) - `processor_config.json` - `statistics.json` (dataset normalization stats) - `experiment_cfg/` (training config snapshot) DeepSpeed ZeRO partition states, optimizer, scheduler and RNG files are intentionally omitted — they are needed only to resume training and would add several GB per checkpoint. ## Evaluation **Why `checkpoint-20000` is the goal.** On the 120,017-frame merged set, 20,000 steps is **~8.0 epochs** — **below** v2's validated sweet spot (~15.3 epochs), because the dataset is 2.4× larger than v5's. The live risk here is therefore **underfit**, not overfit. Only `checkpoint-20000` is published (one checkpoint per repo, lineage storage policy); it is tested on the real robot first. The production checkpoint is chosen by **closed-loop** comparison, not by training loss. There is **no held-out split**; train loss is a fit probe, **not** a generalization measure. **Extension to 40k — no longer a true-resume.** A clean continuation from `checkpoint-20000` to 40,000 steps would have produced `checkpoint-30000` (~12 ep) and `checkpoint-40000` (~16 ep), bracketing v2's sweet spot. However, the cluster training state (DeepSpeed/optimizer/scheduler) was cleaned from NFS on 2026-06-24, so a true-resume is **no longer possible** — reaching 40k now requires re-training from scratch (`MAX_STEPS=40000`). The published deploy weights are unaffected (eval/inference unchanged). **Closed-loop evaluation — completed (real G1).** `checkpoint-20000` was run closed-loop on the physical G1 + SONIC stack and measured against the **v2 `checkpoint-20000` champion**. Low BC loss is a healthy prerequisite, **not** evidence of a good policy — this closed-loop result is the load-bearing signal. **Procedure.** The bottle was placed at **12 fixed positions** spanning one half-side of the table (the table is fixed relative to the robot). These are the **same 12 positions used for data collection** (see the [source dataset card](https://huggingface.co/datasets/cloudwalk-research/gr00t-g1-grab-bottle-right-hand-105ep-v1)). **One trial per position** (12 trials) gives the grid result. Legend: 🟥 red = knocked the bottle over · 🟧 orange = stuck in an "indecision" loop, never reaching the bottle · 🟩 green = successfully grasped the bottle. ![Closed-loop eval: 12-position grid, checkpoint-20000](eval_12_positions_20k.png) **Results — checkpoint-20000.** Grid (12 fixed positions, 1 trial each): | Outcome | Count | Rate | |---|---|---| | 🟩 Success | 8/12 | 67% | | 🟧 Non-convergence (indecision loop) | 3/12 | 25% | | 🟥 Error (knocked bottle over) | 1/12 | 8% | **Conclusion.** More data did not beat better data. v6 trains on the largest set (502 episodes / 120,017 frames, 2.4× v5) yet lands below both v2 (10/12) and v5 (9/12). Its bright spot is the tied-lowest knock-over rate (1/12, 8%, tied with v2) — the merged set's worst-position coverage taught it to avoid toppling the bottle — but the highest non-convergence (3/12, 25%) shows the 2.4× larger set at only ~8 epochs *underfit*: the policy often can't commit. The lineage already flagged underfit as the live risk; the closed-loop result confirms it. **v2 remains in production**. ## How to download ```python from huggingface_hub import snapshot_download local = snapshot_download( repo_id="cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-502ep-v6-finetune", repo_type="model", allow_patterns=["checkpoint-20000/*"], ) print(local) ``` ## How to deploy Start the GR00T policy server (from an [Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T) environment) pointing at the downloaded checkpoint: ```bash python -m gr00t.eval.run_gr00t_server \ --model-path /checkpoint-20000 \ --embodiment-tag UNITREE_G1_SONIC \ --device cuda:0 --host 0.0.0.0 --port 5550 ``` Closed-loop control of the G1 (sim or real) is driven by the **SONIC whole-body controller** in [GR00T-WholeBodyControl](https://github.com/NVlabs/GR00T-WholeBodyControl): the policy emits `motion_token` + hand-joint targets that the SONIC WBC decodes into whole-body joint commands. The server must be launched with the **same `UNITREE_G1_SONIC` embodiment tag** used in training. See the NVlabs [VLA inference tutorial](https://nvlabs.github.io/GR00T-WholeBodyControl/tutorials/vla_inference.html). This checkpoint is **not** plug-and-play on hardware — it requires the SONIC C++ deploy stack and the matching G1 setup. ## Known caveats 1. **Right-hand-only, single task.** The dataset is one task ("grab the bottle") executed with the right hand. Left-hand and locomotion action dims reflect the (largely stationary) demonstrations; do not expect bimanual or locomotion behavior. 2. **Single camera.** Only the `ego_view` (head) camera was recorded — no wrist cameras. The policy has never seen wrist views. 3. **Deployment needs the SONIC stack.** The checkpoint outputs `motion_token` + hand joints for `UNITREE_G1_SONIC`; it only produces robot motion through the SONIC WBC + ZMQ deploy pipeline. It is not directly executable on a bare G1. 4. **Train loss is not held-out.** No episode split; the loss is a smoothed train-fit probe, not a generalization metric. 5. **Underfit risk (not overfit).** 20,000 steps = ~8.0 epochs on 120,017 frames — *below* v2's validated ~15.3-epoch sweet spot, because the dataset is 2.4× larger. The live risk is under-training, not overfitting. `checkpoint-20000` (~8.0 ep) is the goal; reaching 40k (~16 ep) is now a from-scratch re-train (the resume state was cleaned from NFS 2026-06-24) — see [Evaluation](#evaluation). 6. **Single published checkpoint.** Only `checkpoint-20000` is published (one checkpoint per repo, lineage storage policy). There is no intermediate rung — a 40k extension (now only via from-scratch re-train) would add `checkpoint-30000` and `checkpoint-40000`. 7. **Merged dataset provenance.** v6's data is a merge of two source sets (the 105-ep set + a new 115-ep "worst-positions" set), re-curated at radius 20 with grasp-frame preservation (frames near the grasp are no longer removed) and episodes 95–114 kept as-is (482 curated segments + 20 as-is = 502). The grasp-frame fix is a behavioral improvement over earlier versions whose effect is to be validated in closed-loop. 8. **Training environment.** Trained on a shared B200 node with `NCCL_IB_DISABLE=1 NCCL_P2P_LEVEL=NVL` (InfiniBand off, P2P over NVLink) and W&B in offline mode (synced post-run). These affect only the training run, not the weights. ## Lineage | Version | Dataset | Episodes | Frames | Epochs @20k | Closed-loop | Notes | | --- | --- | --- | --- | --- | --- | --- | | [v1](https://huggingface.co/cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-105ep-v1-finetune) | [105ep-v1](https://huggingface.co/datasets/cloudwalk-research/gr00t-g1-grab-bottle-right-hand-105ep-v1) | 105 | 70,680 | 13.6 | ✅ 4/12 (33%) | First GR00T N1.7 + SONIC fine-tune at CW-RL; right-hand bottle pick. | | [v2](https://huggingface.co/cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-210ep-v2-finetune) | [right-hand-v2](https://huggingface.co/datasets/cloudwalk-research/gr00t-g1-grab-bottle-right-hand-v2) | 210 | 62,772 | 15.3 | ✅ 10/12 (83%) — **champion** | Curated (episodes split into shorter windows, bad segments removed). **`checkpoint-20000` = current production champion (10/12).** | | [v4](https://huggingface.co/cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-417ep-v4-finetune) | [radius-5](https://huggingface.co/datasets/cloudwalk-research/gr00t-g1-grab-bottle-right-hand-zero-wandering-smooth-radius-5) | 417 | 48,577 | 19.8 | ✅ 6/12 (50%) | Zero-wandering, **most aggressive** curation (radius 5 → short windows). | | [v5](https://huggingface.co/cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-314ep-v5-finetune) | [radius-20](https://huggingface.co/datasets/cloudwalk-research/gr00t-g1-grab-bottle-right-hand-zero-wandering-smooth-radius-20) | 314 | 50,496 | 19.0 | ✅ 9/12 (75%) | Zero-wandering, **least aggressive** curation (radius 20 → longer windows, ~161 f/ep). | | **v6 (this)** | [radius-20-merged](https://huggingface.co/datasets/cloudwalk-research/gr00t-g1-grab-bottle-right-hand-radius-20-merged) | 502 | 120,017 | 8.0 | ✅ 8/12 (67%) | **Merged** (105-ep + 115-ep "worst-positions"), radius-20 with grasp-frame preservation. Largest set → 20k = only ~8 ep (underfit risk). | | [v7](https://huggingface.co/cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-220ep-v7-finetune) | [speedup-3mm-v1](https://huggingface.co/datasets/cloudwalk-research/gr00t-g1-grab-bottle-right-hand-speedup-3mm-v1) | 220 | 60,163 | ~16.0 | ⏳ TBD | **DP speedup resampling** (wrist-Cartesian, 3 mm/frame, max_K=40) of the merged set; **no segment removal** — reference baseline. ≈16 ep @20k lands on v2's sweet spot. | ## References - **GR00T N1** — Open foundation model for generalist humanoid robots; base policy fine-tuned here. [Paper (arXiv:2503.14734)](https://arxiv.org/abs/2503.14734), [base model](https://huggingface.co/nvidia/GR00T-N1.7-3B). - **Isaac-GR00T** — Training / inference / deployment stack used for this fine-tune. [GitHub](https://github.com/NVIDIA/Isaac-GR00T). - **GR00T-WholeBodyControl (SONIC)** — Whole-body controller + teleop + VLA deployment for the G1. [GitHub](https://github.com/NVlabs/GR00T-WholeBodyControl), [SONIC paper (arXiv:2511.07820)](https://arxiv.org/abs/2511.07820). - **Training dataset** — [`cloudwalk-research/gr00t-g1-grab-bottle-right-hand-radius-20-merged`](https://huggingface.co/datasets/cloudwalk-research/gr00t-g1-grab-bottle-right-hand-radius-20-merged) (CloudWalk Research, 2026), PICO 4 Ultra teleoperation on the Unitree G1 with SONIC WBC; a merge of the 105-ep set + a new 115-ep "worst-positions" set, re-curated at radius 20 with grasp-frame preservation (502 episodes, 120,017 frames). ## Attribution Developed by **cloudwalk-research** in the **CloudWalk Robotics Lab (CW-RL)**. Fine-tuned from [`nvidia/GR00T-N1.7-3B`](https://huggingface.co/nvidia/GR00T-N1.7-3B) using [Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T); targets the Unitree G1 with the [SONIC](https://github.com/NVlabs/GR00T-WholeBodyControl) whole-body controller. ## Citation ```bibtex @misc{cwrl_gr00t_grab_bottle_v6_2026, title = {GR00T N1.7 Fine-Tune v6 --- Unitree G1 "grab the bottle" (right hand, SONIC WBC, merged radius-20 curated 502-ep dataset)}, author = {{CloudWalk Robotics Lab}}, year = {2026}, howpublished = {Hugging Face model repository}, url = {https://huggingface.co/cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-502ep-v6-finetune} } ``` ## License Inherits the **NVIDIA Open Model License Agreement** of the base model [`nvidia/GR00T-N1.7-3B`](https://huggingface.co/nvidia/GR00T-N1.7-3B) — see the [license terms](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/). This is a research preview: not intended for safety-critical use; closed-loop deployment on a physical humanoid requires human oversight.