|
|
| # LingBot-VLA-V2: From Foundation to Application |
|
|
| <p align="center"> |
| <img src="assets/lingbot_vla2_framework.png" width="100%"> |
| </p> |
|
|
| --- |
|
|
| **LingBot-VLA-V2** is a practical Vision-Language-Action foundation model designed to move from large-scale pre-training toward reliable real-world robot applications. |
|
|
| Compared with LingBot-VLA, V2 improves three core capabilities: |
|
|
| - **Generalization across tasks and embodiments**: a redesigned data pipeline curates around **60,000 hours** of pre-training data, including **50,000 hours** of robot trajectories across **20 robot configurations** and **10,000 hours** of egocentric human videos. |
| - **Expanded action space**: the unified representation supports arms, end-effectors, grippers, dexterous hands, waist, head, and mobile-base signals instead of only standard dual-arm manipulation. |
| - **Predictive dynamics modeling**: future prediction is used as a proxy task, with DINO-Video providing semantic temporal priors and LingBot-Depth providing geometric cues. |
|
|
|
|
| ### MoE Action Expert |
|
|
| To improve cross-embodiment scaling, LingBot-VLA-V2 uses sparse MoE layers inside the action expert. Fine-grained expert segmentation and shared expert isolation allow universal priors and specialized embodiment/task patterns to coexist under the same active compute budget. |
|
|
| <p align="center"> |
| <img src="assets/lingbot_vla2_loss_mse_comparison.png" width="90%"> |
| </p> |
|
|
| ### Dual-Query Distillation |
|
|
| LingBot-VLA-V2 appends current and future perceptual queries to the visual/text tokens. These queries are distilled from LingBot-Depth and DINO-Video, encouraging causal inference to capture both current scene geometry and future scene evolution. |
|
|
| <p align="center"> |
| <img src="assets/lingbot_vla2_vis_distillation.png" width="100%"> |
| </p> |
|
|
| --- |
|
|
| ## Model Sources |
|
|
| - Paper: From Foundation to Application: Improving VLA Models in Practice |
| - Repository: https://github.com/robbyant/lingbot-vla-v2 |
| - Project Page: https://technology.robbyant.com/lingbot-vla-v2 |
|
|
|
|
| --- |
|
|
| ## License Agreement |
| This project is licensed under the [Apache-2.0 License](LICENSE). |
|
|
| ## Acknowledgement |
| This codebase is builded on the [VeOmni](https://arxiv.org/abs/2508.02317) project. Thanks for their excellent work! |
