atharva-pantheon/act-pantheon-yam-screwdriver-vn

ACT — Pantheon YAM 'swap screwdriver head' — Velocity-Normalized

ACT (Action-Chunking Transformer) policy for a bimanual YAM manipulation task, trained as one arm of a Velocity-Normalization (VN) ablation. This checkpoint: VN.

Trained on velocity-normalized demonstrations (VN re-times each demo to a consistent speed profile: idle removed, fast motion slowed, cruising aligned).

Companion model: atharva-pantheon/act-pantheon-yam-screwdriver-naive — the other arm of the ablation (same data, same config, uniform 30→10 Hz downsample).

---

Research summary

Question. Does Velocity-Normalization (VN) preprocessing — re-timing teleop demos so a policy sees a consistent end-effector speed distribution — change what an ACT policy learns, holding the underlying demonstrations fixed?

Setup. A controlled ablation: two ACT policies, identical architecture, hyperparameters, and seed; the only difference is how source frames are selected when building the 10 Hz training set.

	this model (VN)	companion
frame selection	velocity-normalized 30→10 Hz	uniform 30→10 Hz downsample

Data

• Task: "swap the tool head on the screwdriver" (task_index 16) — 379 episodes, ~5.2 h, the task with the most episodes. Bimanual YAM, 14-D joints (zero-padded to 20), 3 cameras (top / wrist-L / wrist-R), 30 fps.
• Both training sets built at 224×224, 10 Hz, same writer, same action labeling. VN frame selection is the sole variable.

Velocity Normalization (VN)

Implementation: https://github.com/vovw/vn-pipeline. End-effector speed via forward kinematics (pinocchio, YAM URDF link_6), bimanual speed = max over arms. Two stages:

• Stage 1 (inter-episode): align each episode's cruising speed (30th-pct) toward the median (clamp 0.75–1.5×).
• Stage 2 (intra-episode): a smooth monotonic speed map H(s) that slows the fast tail; gripper-event windows and trailing idle preserved.

Applied to this task: breakpoints m=0.024, M=0.162 m/s; 566,204 source frames → 211,501 VN frames (1.12× duration ratio; 1.1% idle dropped; Stage-1 factor median 1.0×, range 0.75–1.5). The naive baseline is a plain uniform 3× downsample (≈189k frames, idle kept).

Training

ACT, ResNet18 (ImageNet) vision backbone, chunk_size=30, n_action_steps=30, n_obs_steps=1, 224×224, batch 64, lr 1e-5, 10,000 steps (≈6.7 epochs), seed 1000. A100 80 GB, both runs concurrent. Logged to W&B project vn-act-screwdriver.

Result

run	start loss	final loss (L1+KL)
VN	5.27	0.294
naive (no-VN)	5.19	0.310

Both converge tightly (same demonstrations). Training loss is not the verdict — VN's intended benefit is consistent execution speed at inference, which requires a robot/sim rollout to evaluate. What this run establishes: both policies train cleanly to convergence on identical data with VN frame-selection as the only difference.

Files

• model.safetensors — ACT weights (~52 M params)
• config.json, train_config.json — policy + training config
• policy_preprocessor*/policy_postprocessor* — input/output normalization (required to run)
• comparison.png — VN vs no-VN training-loss curves

Usage (LeRobot)

from lerobot.policies.act.modeling_act import ACTPolicy
policy = ACTPolicy.from_pretrained("atharva-pantheon/act-pantheon-yam-screwdriver-vn")

Notes / provenance

• State/action are 20-D (14-D YAM joints zero-padded; valid_action_dims=14).
• Built via a v2.0→v3.0 adapter (the vn-pipeline run_vn.py targets v3.0 input).
• Engineering note: on the 128-core training box, capping OMP_NUM_THREADS was essential — uncapped, ToTensor cost 236 ms/image (thread-dispatch overhead) vs 0.27 ms capped (1000×), which otherwise starved the GPU.