README.md

---
license: apache-2.0
library_name: pytorch
tags:
- vitra
- opentouch
- tactile
- action-prediction
- hand-pose
---

# OpenTouch-Adapted VITRA Step140000

This repository contains the 10k-step OpenTouch adaptation checkpoint for the
VLA-HAND VITRA action model.

The model was initialized from the GigaHands-finetuned VITRA `step140000`
checkpoint and adapted on the OpenTouch VITRA-format train split:

```text
dataset: datasets/vitra_opentouch_keypoint_full_text_aligned
data_mix: opentouch_keypoint_train
train samples: 261,211 frame-level samples
episodes: 2,366
action horizon: 16
action shape: [16, 192]
global_step: 10,000
```

## Files

```text
config.json
epoch=0-step=10000.ckpt/
  weights.pt
  meta.json
```

`optimizer.pt` is intentionally not included. The uploaded files are intended for
loading the adapted model weights for evaluation, action caching, and downstream
touch-editor experiments.

## Usage

Download:

```bash
huggingface-cli download \
  LeoJiangOR/opentouch-vitra-step140000-adapted \
  --local-dir checkpoints/opentouch-vitra-step140000-adapted
```

Then point evaluation or cache-generation scripts at:

```text
checkpoints/opentouch-vitra-step140000-adapted/epoch=0-step=10000.ckpt
```

## Training Method

The OpenTouch editing pipeline uses this adapted VITRA checkpoint as a frozen base
model to generate cached action predictions:

```text
a_base: frozen VITRA prediction, [16, 192]
a_target: OpenTouch converted future action target, [16, 192]
residual_target = a_target - a_base
current_state: [212]
touch_pressure: causal tactile pressure history, [16, 2, 16, 16]
```

The residual touch editor is trained on cached predictions rather than directly
inside VITRA. It predicts an action residual and only edits the not-yet-executed
suffix after `edit_start_idx`; the already executed prefix is kept fixed.

The default large editor cache uses:

```text
train cache: 50,000 random samples from opentouch_keypoint_train
test cache: 10,000 random samples from opentouch_keypoint_test
```

See the VLA-HAND repository README for the full editing pipeline and caveats:

```text
https://github.com/kaichen-z/VLA-HAND
```