Robotics
Transformers
Safetensors
mibot
feature-extraction
vla
vision-language-action
real-time
custom_code
Instructions to use XiaomiRobotics/Xiaomi-Robotics-0-SimplerEnv-Google-Robot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use XiaomiRobotics/Xiaomi-Robotics-0-SimplerEnv-Google-Robot with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("XiaomiRobotics/Xiaomi-Robotics-0-SimplerEnv-Google-Robot", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
Improve model card: add metadata, paper link, and sample usage
Browse filesHi! I'm Niels, part of the community science team at Hugging Face. I've improved the model card for this repository by adding metadata and documentation regarding the model's purpose and usage.
Specifically, this PR:
- Adds the `robotics` pipeline tag.
- Adds `library_name: transformers` to indicate compatibility and enable the "Use in Transformers" button.
- Links the model card to the research paper, project page, and GitHub repository.
- Includes a sample usage code snippet taken directly from the official GitHub README.
Please let me know if you have any questions!
README.md
CHANGED
|
@@ -1,3 +1,106 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: transformers
|
| 4 |
+
pipeline_tag: robotics
|
| 5 |
+
tags:
|
| 6 |
+
- vla
|
| 7 |
+
- vision-language-action
|
| 8 |
+
- real-time
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Xiaomi-Robotics-0
|
| 12 |
+
|
| 13 |
+
**Xiaomi-Robotics-0** is an advanced Vision-Language-Action (VLA) model with 4.7B parameters, specifically engineered for high-performance robotic reasoning and seamless real-time execution.
|
| 14 |
+
|
| 15 |
+
- **Paper:** [Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution](https://huggingface.co/papers/2602.12684)
|
| 16 |
+
- **Project Page:** [xiaomi-robotics-0.github.io](https://xiaomi-robotics-0.github.io/)
|
| 17 |
+
- **Repository:** [GitHub - Xiaomi-Robotics-0](https://github.com/XiaomiRobotics/Xiaomi-Robotics-0)
|
| 18 |
+
|
| 19 |
+
## Model Description
|
| 20 |
+
Xiaomi-Robotics-0 is first pre-trained on large-scale cross-embodiment robot trajectories and vision-language data, providing it with broad and generalizable action-generation capabilities. It utilizes a carefully designed training recipe and deployment strategy to address inference latency, enabling fast and smooth real-time rollouts on consumer-grade GPUs.
|
| 21 |
+
|
| 22 |
+
## Quick Start: Deployment
|
| 23 |
+
|
| 24 |
+
The model is compatible with the Hugging Face `transformers` ecosystem. By leveraging Flash Attention 2 and bfloat16 precision, it can be run efficiently for robotic manipulation tasks.
|
| 25 |
+
|
| 26 |
+
```python
|
| 27 |
+
import torch
|
| 28 |
+
from transformers import AutoModel, AutoProcessor
|
| 29 |
+
|
| 30 |
+
# 1. Load model and processor
|
| 31 |
+
model_path = "XiaomiRobotics/Xiaomi-Robotics-0"
|
| 32 |
+
model = AutoModel.from_pretrained(
|
| 33 |
+
model_path,
|
| 34 |
+
trust_remote_code=True,
|
| 35 |
+
attn_implementation="flash_attention_2",
|
| 36 |
+
dtype=torch.bfloat16
|
| 37 |
+
).cuda().eval()
|
| 38 |
+
processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, use_fast=False)
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
# 2. Construct the prompt with multi-view inputs
|
| 42 |
+
language_instruction = "Pick up the red block."
|
| 43 |
+
instruction = (
|
| 44 |
+
f"<|im_start|>user
|
| 45 |
+
The following observations are captured from multiple views.
|
| 46 |
+
"
|
| 47 |
+
f"# Base View
|
| 48 |
+
<|vision_start|><|image_pad|><|vision_end|>
|
| 49 |
+
"
|
| 50 |
+
f"# Left-Wrist View
|
| 51 |
+
<|vision_start|><|image_pad|><|vision_end|>
|
| 52 |
+
"
|
| 53 |
+
f"Generate robot actions for the task:
|
| 54 |
+
{language_instruction} /no_cot<|im_end|>
|
| 55 |
+
"
|
| 56 |
+
f"<|im_start|>assistant
|
| 57 |
+
<cot></cot><|im_end|>
|
| 58 |
+
"
|
| 59 |
+
)
|
| 60 |
+
|
| 61 |
+
# 3. Prepare inputs
|
| 62 |
+
# Assuming `image_base`, `image_wrist`, and `proprio_state` are already loaded
|
| 63 |
+
inputs = processor(
|
| 64 |
+
text=[instruction],
|
| 65 |
+
images=[image_base, image_wrist], # [PIL.Image, PIL.Image]
|
| 66 |
+
videos=None,
|
| 67 |
+
padding=True,
|
| 68 |
+
return_tensors="pt",
|
| 69 |
+
).to(model.device)
|
| 70 |
+
|
| 71 |
+
# Add proprioceptive state and action mask
|
| 72 |
+
robot_type = "libero"
|
| 73 |
+
inputs["state"] = torch.from_numpy(proprio_state).to(model.device, model.dtype).view(1, 1, -1)
|
| 74 |
+
inputs["action_mask"] = processor.get_action_mask(robot_type).to(model.device, model.dtype)
|
| 75 |
+
|
| 76 |
+
# 4. Generate action
|
| 77 |
+
with torch.no_grad():
|
| 78 |
+
outputs = model(**inputs)
|
| 79 |
+
|
| 80 |
+
# Decode raw outputs into actionable control commands
|
| 81 |
+
action_chunk = processor.decode_action(outputs.actions, robot_type=robot_type)
|
| 82 |
+
print(f"Generated Action Chunk Shape: {action_chunk.shape}")
|
| 83 |
+
```
|
| 84 |
+
|
| 85 |
+
## Benchmark Results
|
| 86 |
+
|
| 87 |
+
The model has been evaluated extensively on standard simulation benchmarks:
|
| 88 |
+
|
| 89 |
+
| Benchmark | Description | Performance |
|
| 90 |
+
| :--- | :--- | :--- |
|
| 91 |
+
| **LIBERO** | Fine-tuned on four LIBERO suites. | **98.7%** (Avg Success) |
|
| 92 |
+
| **CALVIN** | Fine-tuned on ABCD→D Split. | **4.80** (Avg Length) |
|
| 93 |
+
| **SimplerEnv** | Fine-tuned on Fractal dataset (Google Robot). | **85.5%** (VM) / **74.7%** (VA) |
|
| 94 |
+
| **SimplerEnv** | Fine-tuned on Bridge dataset (WidowX). | **79.2%** |
|
| 95 |
+
|
| 96 |
+
## Citation
|
| 97 |
+
|
| 98 |
+
```bibtex
|
| 99 |
+
@misc{robotics2026xiaomi,
|
| 100 |
+
title = {Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution},
|
| 101 |
+
author = {Xiaomi Robotics},
|
| 102 |
+
howpublished={\url{https://xiaomi-robotics-0.github.io}},
|
| 103 |
+
year = {2026},
|
| 104 |
+
note={Project Website}
|
| 105 |
+
}
|
| 106 |
+
```
|