NVIDIA Isaac Teleop and GR00T 1.7 Open VLA Model Available in LeRobot

lior ben horin

liorbenhorin-nv

Follow

nvidia

Kartik S

nvkartik

Follow

nvidia

Johnny Nuñez Cano

johnnynv

Follow

nvidia

Edith Llontop

ellontop

Follow

nvidia

Leung

naleung

Follow

nvidia

Andrew C Wrenn

awrenn53

Follow

nvidia

Shane Reetz

sreetz-nv

Follow

nvidia

Introduction

NVIDIA Isaac Teleop and NVIDIA Isaac GR00T 1.7 are now available on LeRobot, bringing new capabilities for open robot learning and humanoid development.

GR00T 1.7 is the latest open, commercially viable Vision-Language-Action (VLA) foundation model for general-purpose humanoid robots. It delivers improved performance and expanded manipulation capabilities over GR00T N1.5. With this release, GR00T 1.7 replaces N1.5 in LeRobot, and developers should transition to the new model, as N1.5 is no longer supported.

The Isaac GR00T 1.7 model is also available on the NVIDIA Isaac GR00T open development platform, accelerating end-to-end humanoid development and coming pre-integrated with the GR00T reference humanoid robot, an open reference design for academic research.

Isaac Teleop is a teleoperation framework for collecting robot demonstration data. It enables developers to capture high-quality demonstrations, real and sim, in formats compatible with downstream training pipelines, streamlining the workflow from data collection to imitation learning and policy fine-tuning.

GR00T 1.7 on LeRobot

GR00T 1.7 is now available on LeRobot, making it faster and easier to train, fine-tune, and deploy NVIDIA’s latest Vision-Language-Action foundation model. Developers can use LeRobot’s open workflows to quickly adapt the model to their own tasks and robots.

Models trained anywhere can also be deployed through LeRobot while maintaining the same benchmarked performance, providing a seamless path from development to deployment.

The guide below will walk you through installation, training and deployment with GR00T 1.7, via LeRobot - of a real world task executed with the SO-101 robot

Installation

Running GR00T via LeRobot is very simple. Before starting, verify you meet the GR00T prerequisites, everything else will be handled by the LeRobot installation. To follow this guide, we assume you have your SO101 Robot configured, calibrated and ready to work for you.

The following commands will install LeRobot and all the dependencies required for GR00T training and inference

# Install UV if not already installed, then start a new shell
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install ffmpeg
sudo apt install ffmpeg

# From you working directory
git clone https://github.com/huggingface/lerobot.git
cd lerobot

# Install LeRobot,GR00T and dependecnies
uv venv --python 3.12
uv pip install -e ".[groot,training,feetech,viz]"

DGX Spark users you should install compatible CUDA 13 torch version, like so

uv pip install "torch==2.11.0+cu130" "torchvision==0.26.0+cu13" --index-url https://download.pytorch.org/whl/cu130
# Force uv to not rollback torch for the rest of the session
export UV_NO_SYNC=1 

Isaac Teleop Data collection

There are two ways to collect data, either using a VR headset or the SO-101 leader arm.

• For the SO-101 leader arm follow these instructions, SO-101 Teleoperation documentation.
• For using a VR headset follow the guide here. The steps below give an overview of the commands you will be running.

1.Run the following to install Isaac Teleop for data collection.

# LeRobot with the extras this example uses:
#   feetech    - SO-101 serial motor bus
#   kinematics - Placo IK solver (XR controller path)
#   dataset    - dataset recording (record.py)
# huggingface_hub >= 1.5 is needed by the automatic URDF fetch (Buckets API).
uv pip install -e ".[feetech,kinematics,dataset]" "huggingface_hub>=1.5"

# Isaac Teleop from public PyPI. `cloudxr` brings the CloudXR runtime bindings;
# `retargeters-lite` is the scipy-based retargeter path that resolves on both
# x86_64 and ARM (the full `retargeters` extra does not resolve on aarch64).
uv pip install "isaacteleop[cloudxr,retargeters-lite]~=1.3.131" "scipy>=1.14"

2. Setup your environment variables.

# make sure that camera index and robot id and port matches your setup! 
# find index using `uv run lerobot-find-cameras opencv`
WRIST_CAM='wrist: {type: opencv, index_or_path: 2, width: 640, height: 480, fps: 30, fourcc: "MJPG"}'
FRONT_CAM='front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30, fourcc: "MJPG"}'
export ROBOT_CAMERAS="{ $WRIST_CAM, $FRONT_CAM }"
export ROBOT_ID=follower_robot
export ROBOT_PORT=/dev/ttyACM0

3. Test your teleoperation setup with your headset.

uv run python examples.isaac_teleop_to_so101.teleoperate \
    --robot.type=so101_follower \
    --robot.port=$ROBOT_PORT \
    --robot.id=$ROBOT_ID \
    --teleop.type=xr_controller

4. Collect data!

export DATASET_NAME=your_data_set
export HF_USER=your_hf_username
export DATASET=$HF_USER/$DATASET_NAME

uv run python examples.isaac_teleop_to_so101.record \
    --robot.type=so101_follower \
    --robot.port=$ROBOT_PORT \
    --robot.id=$ROBOT_ID \
    --teleop.type=xr_controller \
    --robot.cameras=$ROBOT_CAMERAS \
    --dataset.repo_id=$DATASET \
    --dataset.single_task="Pick up vial and put it into the rack" \
    --dataset.num_episodes=50 \
    --dataset.push_to_hub=true

This will create a LeRobot dataset and push it to the Hugging Face Hub, ready to train a policy using GR00T 1.7

Training

With the dataset you collected, you can begin post-training GR00T 1.7. We have been using an RTX 6000 Pro for our testing. Make sure to adapt the command for the dataset name you collected!

uv run hf auth login
uv run wandb login

export DATASET_NAME=your_data_set
export HF_USER=your_hf_username
export DATASET=$HF_USER/$DATASET_NAME
export REPO_ID="${DATASET}_GR00T17" #this is the model that will be uploaded to huggingface
export OUTPUT_DIR=outputs/train/$REPO_ID

uv run lerobot-train \
  --dataset.repo_id=$DATASET \
  --dataset.image_transforms.enable=true \
  --policy.type=groot \
  --policy.device=cuda \
  --policy.base_model_path=nvidia/GR00T-N1.7-3B \
  --policy.embodiment_tag=new_embodiment \
  --policy.chunk_size=16 \
  --policy.n_action_steps=16 \
  --policy.use_relative_actions=true \
  --policy.relative_exclude_joints='["gripper"]' \
  --policy.use_bf16=true \
  --policy.push_to_hub=true \
  --policy.repo_id=$REPO_ID \
  --seed=42 \
  --batch_size=64 \
  --steps=20000 \
  --save_checkpoint=true \
  --save_freq=5000 \
  --use_policy_training_preset=true \
  --env_eval_freq=0 \
  --eval_steps=0 \
  --log_freq=10 \
  --output_dir=$OUTPUT_DIR \
  --job_name=$DATASET \
  --wandb.enable=true \
  --wandb.disable_artifact=true

# Multi-GPU:
uv run accelerate launch \
  --multi_gpu \
  --num_processes=$(nvidia-smi -L | wc -l) \
  .venv/bin/lerobot-train \
  # here you add the entire command from the single GPU

If you are using a remote machine via SSH, you should use tmux to prevent the session from being killed when logging out

We have created an NVIDIA Brev LeRobot Launchable to help you train your models faster - Use this one click deploy to get your GPU instance in minutes

Deployment

When the training job is completed, it's time to see the robot perform. We will use lerobot-rollout for that - You will also see a Rerun visualizer to monitor robot actions and observations. To help the robot succeed, ensure the rest pose of the robot is similar to what you have used when recording - eg. when the robot is in the initial crouch position, it might not see enough of the table to remember its task.

export MODEL_ID=your_trained_model_on_huggingface

# make sure that camera index and robot id and port matches your setup! 
# find index using `uv run lerobot-find-cameras opencv`
WRIST_CAM='wrist: {type: opencv, index_or_path: 2, width: 640, height: 480, fps: 30, fourcc: "MJPG"}'
FRONT_CAM='front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30, fourcc: "MJPG"}'
export ROBOT_CAMERAS="{ $WRIST_CAM, $FRONT_CAM }"
export ROBOT_ID=follower_robot
export ROBOT_PORT=/dev/ttyACM0

uv run lerobot-rollout \
  --strategy.type=base \
  --policy.path=$MODEL_ID \
  --policy.base_model_path=nvidia/GR00T-N1.7-3B \
  --policy.n_action_steps=8 \
  --robot.type=so101_follower \
  --robot.port=$ROBOT_PORT \
  --robot.id=$ROBOT_ID \
  --robot.cameras="$ROBOT_CAMERAS" \
  --task="place the vial in the rack" \
  --duration=60 \
  --device=cuda \
  --display_data=true \
  --inference.type=rtc \
  --inference.rtc.enabled=false \
  --inference.rtc.execution_horizon=10 \
  --inference.queue_threshold=0

LIBERO Benchmark Results

We deployed GR00T 1.7 policy against the LIBERO Benchmark. LIBERO (Lifelong Benchmark for RObot learning, NeurIPS 2023) is a suite of 130 language-annotated tabletop manipulation tasks designed to isolate what kind of generalization a policy needs to handle. Each suite holds all other variables fixed and varies exactly one axis. Below is also a table comparing the performance difference between GR00T 1.7 and GR00T 1.5, showing a strong improvement trend.

Get Started Today

Key Takeaways

• GR00T 1.7 integrates natively into LeRobot, requiring no infrastructure changes for teams already using the framework
• Both the LeRobot and OSS paths use the same nvidia/GR00T-N1.7 weights - pick the workflow that fits your stack
• Post-training requires LeRobot Dataset v3.0 format

Additional Resources

• GR00T 1.7 model on Hugging Face
• GR00T 1.7 GitHub
• LeRobot Documentation
• NVIDIA Isaac GR00T developer portal
• Isaac Teleop Documentation with LeRobot ad SO-101

Stay up-to-date with the latest developments by following NVIDIA on Hugging Face.