nielsr HF Staff commited on
Commit
4390121
·
verified ·
1 Parent(s): 43419fa

Improve model card: add metadata, paper link, and sample usage

Browse files

Hi! I'm Niels, part of the community science team at Hugging Face. I've improved the model card for this repository by adding metadata and documentation regarding the model's purpose and usage.

Specifically, this PR:
- Adds the `robotics` pipeline tag.
- Adds `library_name: transformers` to indicate compatibility and enable the "Use in Transformers" button.
- Links the model card to the research paper, project page, and GitHub repository.
- Includes a sample usage code snippet taken directly from the official GitHub README.

Please let me know if you have any questions!

Files changed (1) hide show
  1. README.md +106 -3
README.md CHANGED
@@ -1,3 +1,106 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: transformers
4
+ pipeline_tag: robotics
5
+ tags:
6
+ - vla
7
+ - vision-language-action
8
+ - real-time
9
+ ---
10
+
11
+ # Xiaomi-Robotics-0
12
+
13
+ **Xiaomi-Robotics-0** is an advanced Vision-Language-Action (VLA) model with 4.7B parameters, specifically engineered for high-performance robotic reasoning and seamless real-time execution.
14
+
15
+ - **Paper:** [Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution](https://huggingface.co/papers/2602.12684)
16
+ - **Project Page:** [xiaomi-robotics-0.github.io](https://xiaomi-robotics-0.github.io/)
17
+ - **Repository:** [GitHub - Xiaomi-Robotics-0](https://github.com/XiaomiRobotics/Xiaomi-Robotics-0)
18
+
19
+ ## Model Description
20
+ Xiaomi-Robotics-0 is first pre-trained on large-scale cross-embodiment robot trajectories and vision-language data, providing it with broad and generalizable action-generation capabilities. It utilizes a carefully designed training recipe and deployment strategy to address inference latency, enabling fast and smooth real-time rollouts on consumer-grade GPUs.
21
+
22
+ ## Quick Start: Deployment
23
+
24
+ The model is compatible with the Hugging Face `transformers` ecosystem. By leveraging Flash Attention 2 and bfloat16 precision, it can be run efficiently for robotic manipulation tasks.
25
+
26
+ ```python
27
+ import torch
28
+ from transformers import AutoModel, AutoProcessor
29
+
30
+ # 1. Load model and processor
31
+ model_path = "XiaomiRobotics/Xiaomi-Robotics-0"
32
+ model = AutoModel.from_pretrained(
33
+ model_path,
34
+ trust_remote_code=True,
35
+ attn_implementation="flash_attention_2",
36
+ dtype=torch.bfloat16
37
+ ).cuda().eval()
38
+ processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, use_fast=False)
39
+
40
+
41
+ # 2. Construct the prompt with multi-view inputs
42
+ language_instruction = "Pick up the red block."
43
+ instruction = (
44
+ f"<|im_start|>user
45
+ The following observations are captured from multiple views.
46
+ "
47
+ f"# Base View
48
+ <|vision_start|><|image_pad|><|vision_end|>
49
+ "
50
+ f"# Left-Wrist View
51
+ <|vision_start|><|image_pad|><|vision_end|>
52
+ "
53
+ f"Generate robot actions for the task:
54
+ {language_instruction} /no_cot<|im_end|>
55
+ "
56
+ f"<|im_start|>assistant
57
+ <cot></cot><|im_end|>
58
+ "
59
+ )
60
+
61
+ # 3. Prepare inputs
62
+ # Assuming `image_base`, `image_wrist`, and `proprio_state` are already loaded
63
+ inputs = processor(
64
+ text=[instruction],
65
+ images=[image_base, image_wrist], # [PIL.Image, PIL.Image]
66
+ videos=None,
67
+ padding=True,
68
+ return_tensors="pt",
69
+ ).to(model.device)
70
+
71
+ # Add proprioceptive state and action mask
72
+ robot_type = "libero"
73
+ inputs["state"] = torch.from_numpy(proprio_state).to(model.device, model.dtype).view(1, 1, -1)
74
+ inputs["action_mask"] = processor.get_action_mask(robot_type).to(model.device, model.dtype)
75
+
76
+ # 4. Generate action
77
+ with torch.no_grad():
78
+ outputs = model(**inputs)
79
+
80
+ # Decode raw outputs into actionable control commands
81
+ action_chunk = processor.decode_action(outputs.actions, robot_type=robot_type)
82
+ print(f"Generated Action Chunk Shape: {action_chunk.shape}")
83
+ ```
84
+
85
+ ## Benchmark Results
86
+
87
+ The model has been evaluated extensively on standard simulation benchmarks:
88
+
89
+ | Benchmark | Description | Performance |
90
+ | :--- | :--- | :--- |
91
+ | **LIBERO** | Fine-tuned on four LIBERO suites. | **98.7%** (Avg Success) |
92
+ | **CALVIN** | Fine-tuned on ABCD→D Split. | **4.80** (Avg Length) |
93
+ | **SimplerEnv** | Fine-tuned on Fractal dataset (Google Robot). | **85.5%** (VM) / **74.7%** (VA) |
94
+ | **SimplerEnv** | Fine-tuned on Bridge dataset (WidowX). | **79.2%** |
95
+
96
+ ## Citation
97
+
98
+ ```bibtex
99
+ @misc{robotics2026xiaomi,
100
+ title = {Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution},
101
+ author = {Xiaomi Robotics},
102
+ howpublished={\url{https://xiaomi-robotics-0.github.io}},
103
+ year = {2026},
104
+ note={Project Website}
105
+ }
106
+ ```