witsense-ai/so101_act_fewshot

ACT — SO-101 Ring Pick-and-Place

An ACT (Action Chunking Transformer) policy trained with LeRobot to pick up a ring and place it on a toy, using an SO-101 follower arm.

Task: pickup the ring and place it on the toy

Model

• Architecture: ACT (~52M parameters), action chunk size 100
• Robot: SO-101 follower — 6 DoF (shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper)
• Cameras: top (overhead, 640×480) and wrist (1280×720)

Training

Trained from scratch with lerobot-train:

• Dataset: witsense-ai/so101_pick_and_place_ring_33 — 33 teleoperated episodes, 18,861 frames @ 30 fps
• Steps: 50,000 · Batch size: 4 · Learning rate: 1e-5
• Final training loss: ~0.11
• Hardware: single RTX 4070 Laptop (8 GB)

lerobot-train \
  --dataset.repo_id=witsense-ai/so101_pick_and_place_ring_33 \
  --policy.type=act \
  --policy.repo_id=witsense-ai/so101_act_fewshot \
  --steps=50000 --batch_size=4

Usage

lerobot-rollout \
  --policy.path=witsense-ai/so101_act_fewshot \
  --robot.type=so101_follower --robot.port=/dev/ttyACM0 \
  --robot.cameras="{top: {type: opencv, index_or_path: /dev/video4, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: /dev/video2, width: 1280, height: 720, fps: 30, fourcc: MJPG}}" \
  --fps=30 --task="pickup the ring and place it on the toy"

Notes

• Real-robot success rate has not yet been formally evaluated.
• The dataset was curated: recorded as 50 episodes, then reduced to 33 after removing episodes where a mid-recording USB re-enumeration swapped the top/wrist camera streams.