---
license: mit
library_name: transformers
tags:
- robotics
- vla
- behavior-1k
- x-vla
base_model: 2toINF/X-VLA-Pt
---

# X-VLA v20 — BEHAVIOR-1K Task 0 (Turn on Radio)

Fine-tune of [`2toINF/X-VLA-Pt`](https://huggingface.co/2toINF/X-VLA-Pt) on a single BEHAVIOR-1K task (turning on the radio receiver), using the **v20 architecture** from [markli1hoshipu/behavior1k-xvla @ v20](https://github.com/markli1hoshipu/behavior1k-xvla/tree/v20).

## Architecture (v20)

- **Additive per-task + per-skill soft prompts** (`task_prompt_hub[task_id] + skill_prompt_hub[skill_id]`), 32 tokens × 1024 dim each, zero-initialized
- **Skill-conditioned progress head** (predicts position within current skill segment)
- **Skill classifier head** on pooled VLM features, trained with sqrt-inverse-frequency weighted CE (λ=0.1, on detached features)
- **Skill-enriched language instructions** at training time, e.g. `"Turn on the radio receiver. Current: move to radio."`
- 23-D action space for the R1Pro robot (3 base-qvel + 4 trunk + 7 arm L + 1 grip L + 7 arm R + 1 grip R), 30-step action horizon, 3 RGB cameras (head + L/R wrist)

## Training

| Setting | Value |
|---|---|
| Base model | `2toINF/X-VLA-Pt` |
| Dataset | [`Hoshipu/behavior-1k-mp-collected-turning-on-radio`](https://huggingface.co/datasets/Hoshipu/behavior-1k-mp-collected-turning-on-radio) (success split) |
| Trainable episodes | 1354 (task 0) |
| Total frames | 2,845,413 |
| GPUs | 8 × NVIDIA H200 |
| Per-GPU batch | 32 |
| Effective batch | 256 |
| Precision | bf16 |
| LR (core) | 5e-5, cosine decay to 5e-6 |
| LR (VLM, soft prompts) | 0.1 × LR (so 5e-6 → 5e-7) |
| Warmup | 2000 steps |
| Freeze schedule | VLM + transformer core frozen for first 1000 steps (only soft prompts + action heads train) |
| Iterations | 60,000 |
| Optimizer | AdamW, betas=(0.9, 0.95), wd=0.0, grad-clip 1.0 |
| Wall-clock | ~14 h 51 m |

### Loss curve (selected steps)

| Step | total | joints | skill_cls | progress |
|---|---|---|---|---|
| 0 | 15.70 | 15.11 | 0.582 | 0.006 |
| 1000 | 0.568 | 0.555 | 0.005 | 0.008 |
| 5000 | 0.107 | 0.101 | 0.000 | 0.006 |
| 10000 | 0.021 | 0.018 | 0.000 | 0.003 |
| 20000 | 0.029 | 0.025 | 0.000 | 0.003 |
| 30000 | 0.023 | 0.020 | 0.000 | 0.003 |
| 40000 | 0.017 | 0.016 | 0.000 | 0.001 |
| 50000 | 0.013 | 0.012 | 0.000 | 0.001 |
| 59980 | 0.014 | 0.011 | 0.000 | 0.003 |

## Usage

```python
from transformers import AutoModel, AutoConfig

config = AutoConfig.from_pretrained("Hoshipu/xvla-v20-task0-mp-radio", trust_remote_code=True)
model  = AutoModel.from_pretrained("Hoshipu/xvla-v20-task0-mp-radio", trust_remote_code=True)
```

Or deploy as an inference WebSocket server (handles all pre/post-processing for OmniGibson observations):

```bash
git clone -b v20 https://github.com/markli1hoshipu/behavior1k-xvla.git
cd behavior1k-xvla
bash setup.sh
cd behavior1k_training
python deploy_b1k.py --model_path <local-snapshot-of-this-repo> --port 8000
```

See [`INFERENCE_README.md`](https://github.com/markli1hoshipu/behavior1k-xvla/blob/v20/behavior1k_training/INFERENCE_README.md) for the protocol.

## Files

- `model.safetensors` — model weights (3.5 GB, bf16)
- `config.json`, `preprocessor_config.json`, `tokenizer*`, `vocab.json`, `merges.txt` — config / tokenizer
- `modeling_xvla.py`, `configuration_xvla.py`, `transformer.py` — **patched** v20 modules (additive task+skill prompts, skill classifier, progress head)
- `modeling_florence2.py`, `configuration_florence2.py`, `action_hub.py`, `processing_xvla.py` — unchanged from base `2toINF/X-VLA-Pt`