Model Usage Inquiry
Hello, thank you for releasing such a great model.
I have a few questions:
(1) With this fine-tuned model in the GR00T N1.7 + SONIC WBC + MuJoCo environment, and it works (showing some grab motion). However, even in an empty MuJoCo space, the robot performs a motion as if trying to grasp something—even when different prompts are used (such as "walk forward", "raise the hand"). Is this the expected behavior?
(2) Do you have any plans to release a fine-tuned model that supports whole-body motion (e.g., grab the bottle and walk forward and place it ...) ?
(3) Is it possible to run the pre-trained model (without 'any' fine-tuning, nvidia/GR00T-N1.7-3B) in the GR00T N1.7 + SONIC WBC ?
I look forward to your reply.
Please feel free to contact me.
E-mail: th0827.park@samsung.com
Best regards,
Taehyun Park
Hello, Taehyun, nice to meet you!
Thanks for testing our model and for your interest in our work. We’re really glad to see that you were able to replicate the checkpoints in simulation.
This v11 model is part of a family of curated datasets based on 220 episodes of teleoperation. However, this is the only model that our team did not fully evaluate on the real G1, so we don’t have test results or specific caveats for this version. I recommend checking the v10 model page (https://huggingface.co/cloudwalk-research/gr00t-n17-g1-grab-bottle-rh-371ep-v10-finetune) for our evaluations and testing details. v10 worked really well!
Regarding your questions:
Unfortunately, we observed the same behavior in our tests across all 10 models. With an empty table, or even with no table at all, the robot still tries to grab an invisible bottle. This happened even after changing the prompt in testing and recording 20 empty scenes with no movement (https://huggingface.co/datasets/cloudwalk-research/gr00t-g1-grab-bottle-right-hand-worst-positions-empty-115ep-v3).
The only alternative we did not explore was collecting teleoperation data using a different prompt from the beginning, since the prompt is provided during data collection. For example: “Grab the bottle if it appears in the scene.” However, gathering enough additional data to test this properly would take a considerable amount of time.We considered creating teleoperation datasets combining manipulation and locomotion to cover the entire task. However, based on the results we obtained from our VLA experiments, we are now moving toward using a decoupled RL policy, where we can use the upper body more freely, while exploring other approaches for executing the task (other than VLA's only).
I’m not sure whether this is possible. We only tested GR00T after applying NVIDIA’s recommended fine-tuning procedure. But that’s a great question — are you planning to test it?
Thanks again for reaching out and for testing our model. Please feel free to ask if you have any other questions!
Best regards,
João Vitor.