---
pipeline_tag: robotics
tags:
- robot manipulation
- multi-modal perception
- vision-language-action
---

# UniLACT: Depth-Aware RGB Latent Action Learning for Vision-Language-Action Models

[**Paper**](https://huggingface.co/papers/2602.20231) | [**Project Page**](https://manishgovind.github.io/unilact-vla/) | [**Code**](https://github.com/ManishGovind/UniLACT)

UniLACT is a transformer-based Vision-Language-Action (VLA) model that incorporates 3D geometric structure through depth-aware latent pretraining. By utilizing UniLARN, a unified latent action learning framework, the model learns a shared embedding space for RGB and depth, enabling downstream policies to inherit stronger spatial priors for precise and contact-rich robot manipulation.

## Abstract
Latent action representations learned from unlabeled videos have recently emerged as a promising paradigm for pretraining vision-language-action (VLA) models without explicit robot action supervision. However, latent actions derived solely from RGB observations primarily encode appearance-driven dynamics and lack explicit 3D geometric structure. To address this limitation, we introduce UniLACT, which incorporates geometric structure through depth-aware latent pretraining. Our proposed UniLARN framework learns a shared embedding space for RGB and depth while explicitly modeling their cross-modal interactions. Extensive experiments demonstrate that UniLACT consistently outperforms RGB-based latent action baselines under both in-domain and out-of-domain pretraining regimes.

## Setup

```bash
conda create -n unilact python=3.10 -y
conda activate unilact

git clone https://github.com/manishgovind/uniact-vla.git
cd UniLACT
pip install -r requirements.txt
```

## Citation

```bibtex
@article{govind2026unilactdepthawarergblatent,
  title= {UniLACT: Depth-Aware RGB Latent Action Learning for Vision-Language-Action Models},
  author= {Manish Kumar Govind and Dominick Reilly and Pu Wang and Srijan Das},
  journal={arXiv preprint arXiv:2602.20231},
  year={2026}
}
```