wsagi/GR00T-N1.7-G1-SONIC-LAFAN

GR00T-N1.7 × GEAR-SONIC — LAFAN whole-body motion for Unitree G1 (flow3)

⚠️ Proof-of-concept (derisk checkpoint) — read first

This model memorizes 8 hand-picked LAFAN windows, 1 episode each. Generalization to new prompts/motions is untested (train-set reconstruction MSE ≈ 0.0026, i.e. memorization, no held-out split). The demo clips below are driven with the tracking deviation/fall terminations disabled so each window plays in full — they show prompt → token tracking, and "physically valid" (never fell) is not the same as "visually faithful". Not a deployable general VLA.

A single GR00T N1.7 vision-language-action model fine-tuned to emit the 64-dim FSQ motion token of the GEAR-SONIC whole-body controller (WBC). One model, one prompt → the frozen SONIC WBC decodes a full-body G1 motion (it supplies balance/recovery). No per-joint behavior cloning.

prompt + ego-cam + proprio → GR00T N1.7 → 64-d token → SONIC.decode(50 Hz) → 29-DoF G1

Variant — LAFAN: trained on wsagi/SONIC-VLA-LAFAN, whose motions are hand-picked windows from the LAFAN1 corpus (Ubisoft). Its sibling wsagi/GR00T-N1.7-G1-SONIC-BonesSeed uses NVIDIA's GEAR-SONIC demo motions — same prompt → token → WBC pipeline, different source.

🔗 Code, training & validation scaffolding: vitorcen/LeSONIC — the SONIC VLA project (full principle + dataset pipeline in doc/sonic_vla_principles.html), part of the vitorcen/isaaclab-experience umbrella.

---

🎥 flow3 — looping demo (main)

fight → run → fight → run → dance → run, looping in one session. The GR00T-predicted tokens of six windows are concatenated and decoded by the WBC (offline playback = guaranteed smooth).

🎥 Per-window closed-loop demos

Fight — combat strikes and combo kicks · block and push-kick · fierce swings

Run — jog forward then run backward · sprint back and forth then backpedal · run in a circle

Dance — moonwalk · spin, step back, and clap

---

How LAFAN motions get in: a physical-validity gate

LAFAN clips are foreign choreography to the frozen WBC, so a clip can't simply be "trained on". Each candidate window is first screened by letting the frozen WBC physically track it and asking "did the robot actually fall?" (root height + torso tilt) — not the release's strict per-frame safety envelope, which terminates on a single fast-strike frame even when the robot never falls. Of 10 hand-picked windows, 2 pass the strict gate but 6 never fall; those 6 (plus 2 trackable prefixes) are what this model is trained on. This is a WBC-capability screen, upstream of the VLA: a token is only worth learning if the WBC can physically realize it.

Contents

Weights	GR00T N1.7 (3B), bf16, 3 safetensors shards (~6.3 GB)
Action	motion_token (64) + left/right hand joints, horizon 40, ABSOLUTE
Inputs	observation.images.ego_view (480×640) + proprioception + task_description (prompt)
Embodiment	unitree_g1_sonic (29-DoF G1)
Trained on	wsagi/SONIC-VLA-LAFAN — 8 windows / 5,777 frames
Base	nvidia/GR00T-N1.7-3B (VLM frozen; DiT head + projector trained)

▶️ How to run

The model emits SONIC tokens; you need the GEAR-SONIC WBC to decode them. See vitorcen/LeSONIC:

# offline (smooth, no server):
bash scripts/gear_sonic_flow3.sh
# live closed-loop (GR00T server in the loop):
GR00T_CKPT=<this checkpoint> bash scripts/gear_sonic_live_demo.sh @flow3

⚠️ Honest scope & limitations

• Memorization, not generalization — 8 windows × 1 ep; no held-out. MSE ≈ 0.0026 is train-set.
• "Never fell" ≠ "looks like the action" — visual fidelity is filtered separately in GUI; some windows (e.g. moonwalk) track loosely. The 2 dropped fight windows fell immediately (bad start pose).
• Demos run with deviation terminations disabled so the full window plays; not a balance/fall test.
• Single-frame obs, no memory — one-shot windows may settle toward standing under live inference.
• Depends on the external SONIC WBC + (for live) a ZMQ server.

Provenance / lineage

LAFAN1 (Ubisoft) → retarget to G1 (MimicKit) → physical-validity screen
   → nvidia/GEAR-SONIC WBC tracks each window, record tokens → wsagi/SONIC-VLA-LAFAN
   → finetune nvidia/GR00T-N1.7-3B → this model

Please cite GEAR-SONIC, BONES-SEED (the WBC's training corpus), and LAFAN1. Use under the NVIDIA Open Model License terms of the upstream release; respect Ubisoft's LAFAN1 license.