LeoJiangOR/vitra-gigahands-joint-kd-student-step50000

VITRA GigaHands Joint-KD Student Step 50000

This repository contains a compressed VITRA student checkpoint trained on the cleaned GigaHands all-cam0 keypoints-MANO setup.

Model

• Architecture: VITRA_EncoderStudent
• VLM backend: DINOv2-base vision encoder + DistilBERT text encoder
• Action head: 6-layer DiT-B-6L
• Action dimension: 192
• Prediction horizon: 16
• Teacher: GigaHands-finetuned VITRA checkpoint at step 140000
• Training step: 50000

Training Objective

The student was trained with joint distillation:

total loss = VLM feature distillation + ground-truth action diffusion loss + action-head KD loss

The action-head KD term matches teacher and student diffusion noise predictions at the same noisy action sample and diffusion timestep.

Evaluation Summary

Evaluated on the cleaned GigaHands test split with 1,495 clips and 23,920 valid bimanual frames.

Model	Action MSE ↓	Left MSE ↓	Right MSE ↓
Base VITRA-3B	16.1358	3.4089	45.2258
Finetuned VITRA step140000 teacher	0.4061	0.4763	0.2456
Joint-KD student step50000	0.4532	0.5367	0.2624

Feature alignment to the step140000 teacher:

Metric	Value
VLM cognition MSE ↓	0.000339
VLM cognition cosine ↑	0.9776

Files

• epoch=0-step=50000.ckpt/weights.pt: model weights
• epoch=0-step=50000.ckpt/meta.json: checkpoint metadata
• config/finetune_distill_step140000_joint_kd_all_cam0_keypoints_mano.json: training/inference config
• logs/loss_curve.csv: training loss curve
• docs/distillation_and_test_time_guidance_report.md: experiment report

Notes

This checkpoint is intended for the local VLA-HAND/VITRA codebase. It is a compressed student model, not the original VITRA-3B checkpoint.