susanping nielsr HF Staff commited on
Commit
3f25b13
·
1 Parent(s): 3932f4f

Add model card for Xiaomi-Robotics-0 (#1)

Browse files

- Add model card for Xiaomi-Robotics-0 (69e20ebffab476bc1f4e66c03254e04f4f3e5a85)
- Resolve merge conflict with main branch (8e20fd71ae95424fc93cfe5c517b9f83a8050e00)


Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +94 -1
README.md CHANGED
@@ -1,7 +1,100 @@
1
  ---
2
  license: apache-2.0
 
3
  pipeline_tag: robotics
4
  tags:
5
  - vision-language-action
6
  - vla
7
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ library_name: transformers
4
  pipeline_tag: robotics
5
  tags:
6
  - vision-language-action
7
  - vla
8
+ - robotics
9
+ ---
10
+
11
+ # Xiaomi-Robotics-0
12
+
13
+ Xiaomi-Robotics-0 is an advanced vision-language-action (VLA) model with 4.7B parameters, specifically engineered for high-performance robotic reasoning and seamless real-time execution. It is pre-trained on large-scale cross-embodiment robot trajectories and vision-language data, enabling generalizable action generation across complex robotic tasks.
14
+
15
+ - **Paper:** [Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution](https://huggingface.co/papers/2602.12684)
16
+ - **Project Page:** [https://xiaomi-robotics-0.github.io/](https://xiaomi-robotics-0.github.io/)
17
+ - **Repository:** [https://github.com/XiaomiRobotics/Xiaomi-Robotics-0](https://github.com/XiaomiRobotics/Xiaomi-Robotics-0)
18
+
19
+ ## Key Features
20
+
21
+ * **Strong Generalization**: Pre-trained on diverse cross-embodiment trajectories to handle complex, unseen tasks.
22
+ * **Real-Time Ready**: Optimized with asynchronous execution to minimize inference latency during real-robot rollouts.
23
+ * **Flexible Deployment**: Fully compatible with the Hugging Face `transformers` ecosystem and optimized for consumer-grade GPUs.
24
+
25
+ ## Sample Usage
26
+
27
+ The model can be loaded and run using the `transformers` library. As the model uses custom code, `trust_remote_code=True` is required.
28
+
29
+ ```python
30
+ import torch
31
+ from transformers import AutoModel, AutoProcessor
32
+
33
+ # 1. Load model and processor
34
+ model_path = "XiaomiRobotics/Xiaomi-Robotics-0"
35
+ model = AutoModel.from_pretrained(
36
+ model_path,
37
+ trust_remote_code=True,
38
+ attn_implementation="flash_attention_2",
39
+ dtype=torch.bfloat16
40
+ ).cuda().eval()
41
+ processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, use_fast=False)
42
+
43
+
44
+ # 2. Construct the prompt with multi-view inputs
45
+ language_instruction = "Pick up the red block."
46
+ instruction = (
47
+ f"<|im_start|>user
48
+ The following observations are captured from multiple views.
49
+ "
50
+ f"# Base View
51
+ <|vision_start|><|image_pad|><|vision_end|>
52
+ "
53
+ f"# Left-Wrist View
54
+ <|vision_start|><|image_pad|><|vision_end|>
55
+ "
56
+ f"Generate robot actions for the task:
57
+ {language_instruction} /no_cot<|im_end|>
58
+ "
59
+ f"<|im_start|>assistant
60
+ <cot></cot><|im_end|>
61
+ "
62
+ )
63
+
64
+ # 3. Prepare inputs
65
+ # Assuming `image_base`, `image_wrist` (PIL images), and `proprio_state` (numpy array) are already loaded
66
+ inputs = processor(
67
+ text=[instruction],
68
+ images=[image_base, image_wrist], # [PIL.Image, PIL.Image]
69
+ videos=None,
70
+ padding=True,
71
+ return_tensors="pt",
72
+ ).to(model.device)
73
+
74
+ # Add proprioceptive state and action mask
75
+ robot_type = "libero" # Choose based on dataset/robot type
76
+ inputs["state"] = torch.from_numpy(proprio_state).to(model.device, model.dtype).view(1, 1, -1)
77
+ inputs["action_mask"] = processor.get_action_mask(robot_type).to(model.device, model.dtype)
78
+
79
+ # 4. Generate action
80
+ with torch.no_grad():
81
+ outputs = model(**inputs)
82
+
83
+ # Decode raw outputs into actionable control commands
84
+ action_chunk = processor.decode_action(outputs.actions, robot_type=robot_type)
85
+ print(f"Generated Action Chunk Shape: {action_chunk.shape}")
86
+ ```
87
+
88
+ ## Citation
89
+
90
+ If you find this project useful, please consider citing:
91
+
92
+ ```bibtex
93
+ @misc{robotics2026xiaomi,
94
+ title = {Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution},
95
+ author = {Xiaomi Robotics},
96
+ howpublished={\url{https://xiaomi-robotics-0.github.io}},
97
+ year = {2026},
98
+ note={Project Website}
99
+ }
100
+ ```