ASethi04/MolmoAct2-BimanualYAM-oranges-12k-v2-nodropout

MolmoAct2-BimanualYAM — "Put all oranges in the bowl" (12k steps, v2 minus residual dropout (A/B partner))

Fine-tune of allenai/MolmoAct2-BimanualYAM on brandonyang/yam-vive-teleop (80 teleop episodes, 74,927 frames, bimanual YAM, 30 fps).

Successor to ASethi04/MolmoAct2-BimanualYAM-oranges-12k: same recipe (12,000 steps ≈ 10.3 epochs, LoRA r=64 on the VLM @ 5e-5, action expert fully fine-tuned @ 1e-4, narrowed color jitter with affine augmentation kept), retrained with the MolmoAct2 author's training fixes from huggingface/lerobot PR#4249:

• flow-matching noise/timesteps/velocity targets kept in fp32 under autocast (previously quantized to bf16)
• distributed-correct weighted-loss normalization (denominator all-reduced across ranks)
• episode-boundary actions supervised on clamped fixed-horizon targets instead of masked out — the policy now learns terminal settling behavior
• input-embedding freeze no longer risks freezing tied output parameters
• llm_residual_dropout=0.1 (original recipe's regularization)

Also new in this checkpoint: inference_action_mode="continuous" is baked into the config — no override needed at serving time.

Trainable params	727,296,544 / 5,591,928,304 (13%)
Steps / epochs	12,000 / 10.25
Global batch	64 (8 GPUs × 8)
Optimizer	AdamW β=(0.9,0.95), ε=1e-6, wd=0, clip 1.0
Schedule	cosine, 600-step warmup, decay ratio 0.1
Precision	bfloat16 + gradient checkpointing
Action mode	both (discrete FAST + flow matching), 8 flow timesteps
Chunk / executed	30 / 30 (1 s @ 30 Hz)
Cameras	observation.images.{top,left,right} @ 480×270
Augmentation	brightness/contrast/saturation 0.8–1.2, hue ±0.02, sharpness 0.5–1.5, affine ±5°/5%
Normalization	quantile q01/q99; grippers raw
Split / seed	100/0 (all 80 episodes) / 1000

Final training step: step:12K smpl:768K ep:820 epch:10.25 loss:0.664 grdn:2.061 lr:5.0e-06 updt_s:2.415 data_s:0.070 smp/s:26 mem_gb:25.86 discrete_ce_loss:0.660 discrete_z_loss:0.000 action_flow_loss:0.003

Deployment notes

1. Do not pass norm_tag — stats are baked into the processor files.
2. Task string must match training exactly: Put all oranges in the bowl.
3. Raise the client gripper rate limit: --robot.max_gripper_delta=0.05 (the data contains gripper commands up to 0.05/tick; the 0.03 default throttles grasps).
4. All three cameras at 16:9 (training was 480×270); the docs' 640×480 top camera is a silent train/deploy mismatch.

lerobot-policy-server \
  --policy.pretrained_name_or_path=ASethi04/MolmoAct2-BimanualYAM-oranges-12k-v2-nodropout \
  --policy.model_dtype=bfloat16 --policy.device=cuda --host=0.0.0.0 --port=8081

Limitations

Single task, 80 demonstrations, no held-out validation set. Validate on hardware with a no-motion action probe before arming the robot.