Hugging Face's logo

Spaces:
Prithwi13
/
pushT_Demo
Running

App Files Community
Prithwi13 commited on
Commit
c5a7b09
·
verified ·
1 Parent(s): 648ec3a

Update index.html

Browse files
Files changed (1) hide show
  1. index.html +242 -19
index.html CHANGED
@@ -1,19 +1,242 @@
1
- <!doctype html>
2
- <html>
3
- <head>
4
- <meta charset="utf-8" />
5
- <meta name="viewport" content="width=device-width" />
6
- <title>My static Space</title>
7
- <link rel="stylesheet" href="style.css" />
8
- </head>
9
- <body>
10
- <div class="card">
11
- <h1>Welcome to your static Space!</h1>
12
- <p>You can modify this app directly by editing <i>index.html</i> in the Files and versions tab.</p>
13
- <p>
14
- Also don't forget to check the
15
- <a href="https://huggingface.co/docs/hub/spaces" target="_blank">Spaces documentation</a>.
16
- </p>
17
- </div>
18
- </body>
19
- </html>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!DOCTYPE html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="UTF-8">
5
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
6
+ <title>LeRobot Diffusion Policy - PushT-v0</title>
7
+ <script src="https://cdn.jsdelivr.net/npm/@tailwindcss/browser@4"></script>
8
+ <link rel="preconnect" href="https://fonts.googleapis.com">
9
+ <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
10
+ <link href="https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet">
11
+ <link href="https://cdnjs.cloudflare.com/ajax/libs/prism/1.29.0/themes/prism-tomorrow.min.css" rel="stylesheet" />
12
+ <style>
13
+ body {
14
+ font-family: 'Inter', sans-serif;
15
+ }
16
+ pre, code {
17
+ font-family: 'JetBrains Mono', monospace !important;
18
+ }
19
+ </style>
20
+ </head>
21
+ <body class="bg-slate-950 text-slate-100 min-h-screen selection:bg-indigo-500 selection:text-white">
22
+
23
+ <header class="border-b border-slate-800 bg-slate-900/50 backdrop-blur sticky top-0 z-50">
24
+ <div class="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 h-16 flex items-center justify-between">
25
+ <div class="flex items-center space-x-3">
26
+ <span class="flex h-3 w-3 relative">
27
+ <span class="animate-ping absolute inline-flex h-full w-full rounded-full bg-emerald-400 opacity-75"></span>
28
+ <span class="relative inline-flex rounded-full h-3 w-3 bg-emerald-500"></span>
29
+ </span>
30
+ <h1 class="text-lg font-semibold tracking-tight text-white">LeRobot PushT Evaluation</h1>
31
+ </div>
32
+ <div class="flex items-center space-x-2">
33
+ <span class="px-2.5 py-1 text-xs font-medium rounded-md bg-indigo-500/10 text-indigo-400 border border-indigo-500/20">Diffusion Policy</span>
34
+ <span class="px-2.5 py-1 text-xs font-medium rounded-md bg-slate-800 text-slate-400 border border-slate-700">Gymnasium</span>
35
+ </div>
36
+ </div>
37
+ </header>
38
+
39
+ <main class="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-10 space-y-12">
40
+
41
+ <div class="grid grid-cols-1 lg:grid-cols-12 gap-8 items-start">
42
+
43
+ <div class="lg:col-span-5 space-y-4">
44
+ <div class="bg-slate-900 border border-slate-800 rounded-xl overflow-hidden shadow-2xl p-4">
45
+ <div class="flex items-center justify-between mb-3 px-1">
46
+ <span class="text-xs font-medium uppercase tracking-wider text-slate-400">Policy Rollout (300 Steps)</span>
47
+ <span class="text-xs text-slate-500">pusht_policy.mp4</span>
48
+ </div>
49
+ <div class="relative aspect-square rounded-lg overflow-hidden bg-white border border-slate-700">
50
+ <video class="w-full h-full object-contain" autoplay loop muted controls playsinline>
51
+ <source src="pusht_policy.mp4" type="video/mp4">
52
+ Your browser does not support the video tag.
53
+ </video>
54
+ </div>
55
+ </div>
56
+ </div>
57
+
58
+ <div class="lg:col-span-7 space-y-6 lg:pt-2">
59
+ <div>
60
+ <h2 class="text-3xl font-bold tracking-tight text-white mb-3">Autonomous Multi-Modal Manipulation</h2>
61
+ <p class="text-slate-400 leading-relaxed">
62
+ This Space showcases a trained <strong>Diffusion Policy</strong> operating within the <code>PushT-v0</code> simulation environment using Hugging Face's <strong>LeRobot</strong> ecosystem. The agent learns multi-modal trajectories to effectively guide the gray T-shaped block completely into the target green silhouette zone.
63
+ </p>
64
+ </div>
65
+
66
+ <div class="grid grid-cols-1 sm:grid-cols-2 gap-4">
67
+ <div class="p-4 bg-slate-900/60 border border-slate-800 rounded-xl">
68
+ <h3 class="text-xs font-semibold text-slate-400 uppercase tracking-wider mb-2">Observation Space</h3>
69
+ <p class="text-sm font-medium text-slate-200">Pixels &amp; Agent Position</p>
70
+ <p class="text-xs text-slate-500 mt-1">Image Shape: (3, 384, 384)</p>
71
+ </div>
72
+ <div class="p-4 bg-slate-900/60 border border-slate-800 rounded-xl">
73
+ <h3 class="text-xs font-semibold text-slate-400 uppercase tracking-wider mb-2">Action Space</h3>
74
+ <p class="text-sm font-medium text-slate-200">2D Continuous Control</p>
75
+ <p class="text-xs text-slate-500 mt-1">End-effector delta position</p>
76
+ </div>
77
+ <div class="p-4 bg-slate-900/60 border border-slate-800 rounded-xl">
78
+ <h3 class="text-xs font-semibold text-slate-400 uppercase tracking-wider mb-2">Model Source</h3>
79
+ <p class="text-sm font-medium text-slate-200">lerobot/diffusion_pusht</p>
80
+ <p class="text-xs text-slate-500 mt-1">Pre-trained Checkpoint via HF Hub</p>
81
+ </div>
82
+ <div class="p-4 bg-slate-900/60 border border-slate-800 rounded-xl">
83
+ <h3 class="text-xs font-semibold text-slate-400 uppercase tracking-wider mb-2">Pipeline Optimization</h3>
84
+ <p class="text-sm font-medium text-emerald-400">Dynamic Buffer Patching</p>
85
+ <p class="text-xs text-slate-500 mt-1">Overrides state dict pos_grid mismatch</p>
86
+ </div>
87
+ </div>
88
+
89
+ <div class="p-4 bg-amber-500/5 border border-amber-500/20 rounded-xl flex space-x-3">
90
+ <svg class="h-5 w-5 text-amber-500 shrink-0 mt-0.5" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor">
91
+ <path stroke-linecap="round" stroke-linejoin="round" d="M12 9v3.75m-9.303 3.376c-.866 1.5.217 3.374 1.948 3.374h14.71c1.73 0 2.813-1.874 1.948-3.374L13.949 3.378c-.866-1.5-3.032-1.5-3.898 0L2.697 16.126zM12 15.75h.007v.008H12v-.008z" />
92
+ </svg>
93
+ <div class="text-xs text-slate-400 leading-relaxed">
94
+ <strong class="text-slate-200 font-medium">Engineering Note:</strong> The default environment instantiation sets the visual input size to 384×384 pixels, forcing the model's position grid (<code class="text-amber-400">pos_grid</code>) to shape <code class="text-amber-400">[144, 2]</code>. To preserve checkpoint compatibility, the execution engine explicitly overwrites and re-registers the pre-trained token configuration <code class="text-slate-300">[9, 2]</code> buffer layout right after model initialization.
95
+ </div>
96
+ </div>
97
+ </div>
98
+ </div>
99
+
100
+ <hr class="border-slate-800" />
101
+
102
+ <div class="space-y-4">
103
+ <div>
104
+ <h2 class="text-xl font-bold text-white tracking-tight">Deployment & Rollout Script</h2>
105
+ <p class="text-sm text-slate-400 mt-1">The clean implementation pipeline used to compile dataset statistics, patch the architecture shapes, step through environment dynamics, and generate the rollout asset.</p>
106
+ </div>
107
+
108
+ <div class="relative bg-slate-900 border border-slate-800 rounded-xl overflow-hidden shadow-xl max-h-[600px] overflow-y-auto">
109
+ <div class="sticky top-0 bg-slate-900 border-b border-slate-800 px-4 py-2 flex items-center justify-between text-xs text-slate-400 z-10">
110
+ <span class="font-mono">run_pusht.py</span>
111
+ <button id="copyBtn" class="hover:text-white transition flex items-center space-x-1 cursor-pointer">
112
+ <svg class="h-3.5 w-3.5" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor">
113
+ <path stroke-linecap="round" stroke-linejoin="round" d="M15.75 17.25v3.375c0 .621-.504 1.125-1.125 1.125h-9.75a1.125 1.125 0 0 1-1.125-1.125V7.875c0-.621.504-1.125 1.125-1.125H6.75a9.06 9.06 0 0 1 1.5.124m7.5 10.376A8.965 8.965 0 0 0 12 12.75a8.965 8.965 0 0 0-3.75 4.625M18 4.75V3.375c0-.621-.504-1.125-1.125-1.125h-9.75a1.125 1.125 0 0 0-1.125 1.125V4.75m12.75 0V19.5a1.125 1.125 0 0 1-1.125 1.125H18M9 4.75v1.5a1.125 1.125 0 0 0 1.125 1.125h3.75A1.125 1.125 0 0 0 15 6.25v-1.5M9 4.75h6" />
114
+ </svg>
115
+ <span>Copy Code</span>
116
+ </button>
117
+ </div>
118
+ <pre class="m-0 p-4 bg-slate-900 text-sm"><code class="language-python" id="codeBlock">import os
119
+ import gymnasium as gym
120
+ import gym_pusht
121
+ import torch
122
+ import imageio
123
+ from huggingface_hub import hf_hub_download
124
+ import safetensors.torch
125
+ from lerobot.policies.diffusion.modeling_diffusion import DiffusionPolicy
126
+ from lerobot.configs.policies import PreTrainedConfig
127
+ from lerobot.policies.factory import make_pre_post_processors
128
+ from lerobot.envs.utils import preprocess_observation
129
+
130
+ def main():
131
+ # 1. Download checkpoint and load config
132
+ print("Downloading config from lerobot/diffusion_pusht...")
133
+ cfg = PreTrainedConfig.from_pretrained('lerobot/diffusion_pusht')
134
+
135
+ # We override the observation.image feature shape to (3, 384, 384) to match the environment defaults,
136
+ # which instantiates the model's pos_grid as [144, 2] instead of [9, 2] (checkpoint size).
137
+ cfg.input_features['observation.image'].shape = (3, 384, 384)
138
+
139
+ # Build the DiffusionPolicy
140
+ print("Building DiffusionPolicy...")
141
+ policy = DiffusionPolicy(cfg)
142
+ print("Initial pos_grid shape in model:", policy.diffusion.rgb_encoder.pool.pos_grid.shape)
143
+
144
+ # Load weights with strict=False
145
+ print("Downloading and loading safetensors model weights...")
146
+ model_file = hf_hub_download(repo_id='lerobot/diffusion_pusht', filename='model.safetensors')
147
+ state_dict = safetensors.torch.load_file(model_file)
148
+ policy.load_state_dict(state_dict, strict=False)
149
+
150
+ # 2. Patch the pos_grid shape mismatch so inference works
151
+ print("Patching the pos_grid shape mismatch...")
152
+ checkpoint_pos_grid = state_dict['diffusion.rgb_encoder.pool.pos_grid']
153
+ policy.diffusion.rgb_encoder.pool.register_buffer('pos_grid', checkpoint_pos_grid)
154
+ print("Patched pos_grid shape in model:", policy.diffusion.rgb_encoder.pool.pos_grid.shape)
155
+
156
+ # Move policy to correct device and set to eval mode
157
+ policy.to(cfg.device)
158
+ policy.eval()
159
+
160
+ # 3. Create preprocessor / postprocessor with the extracted dataset stats
161
+ print("Creating preprocessor and postprocessor...")
162
+ dataset_stats = {
163
+ 'observation.image': {
164
+ 'mean': state_dict['normalize_inputs.buffer_observation_image.mean'],
165
+ 'std': state_dict['normalize_inputs.buffer_observation_image.std'],
166
+ },
167
+ 'observation.state': {
168
+ 'max': state_dict['normalize_inputs.buffer_observation_state.max'],
169
+ 'min': state_dict['normalize_inputs.buffer_observation_state.min'],
170
+ },
171
+ 'action': {
172
+ 'max': state_dict['normalize_targets.buffer_action.max'],
173
+ 'min': state_dict['normalize_targets.buffer_action.min'],
174
+ }
175
+ }
176
+ preprocessor, postprocessor = make_pre_post_processors(cfg, dataset_stats=dataset_stats)
177
+
178
+ # 4. Instantiate the gym environment
179
+ print("Creating PushT environment...")
180
+ env = gym.make('gym_pusht/PushT-v0', render_mode='rgb_array', obs_type='pixels_agent_pos')
181
+
182
+ # Reset env and cache initial frame
183
+ policy.reset()
184
+ obs, info = env.reset()
185
+ frames = [env.render()]
186
+
187
+ # Run rollout for 300 steps
188
+ print("Running 300 steps rollout...")
189
+ for step in range(300):
190
+ # Format observations to LeRobot format
191
+ obs_t = preprocess_observation(obs)
192
+ obs_t = preprocessor(obs_t)
193
+
194
+ # Select action
195
+ with torch.no_grad():
196
+ action = policy.select_action(obs_t)
197
+ action = postprocessor(action)
198
+
199
+ # Extract numpy action and apply to env (drop batch dimension)
200
+ action_numpy = action.to("cpu").numpy()[0]
201
+ obs, reward, terminated, truncated, info = env.step(action_numpy)
202
+
203
+ # Render frame
204
+ frame = env.render()
205
+ frames.append(frame)
206
+
207
+ if terminated or truncated:
208
+ obs, info = env.reset()
209
+
210
+ # Close env
211
+ env.close()
212
+
213
+ # 5. Save the frames as pusht_policy.mp4
214
+ print("Saving video to pusht_policy.mp4...")
215
+ imageio.mimsave("pusht_policy.mp4", frames, fps=10)
216
+ print("Done! Video saved successfully.")
217
+
218
+ if __name__ == "__main__":
219
+ main()</code></pre>
220
+ </div>
221
+ </div>
222
+ </main>
223
+
224
+ <footer class="text-center py-8 text-xs text-slate-600 border-t border-slate-900 mt-12">
225
+ Powered by LeRobot, Gymnasium, and Hugging Face Static Spaces.
226
+ </footer>
227
+
228
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.29.0/components/prism-core.min.js"></script>
229
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.29.0/plugins/autoloader/prism-autoloader.min.js"></script>
230
+
231
+ <script>
232
+ document.getElementById('copyBtn').addEventListener('click', () => {
233
+ const code = document.getElementById('codeBlock').innerText;
234
+ navigator.clipboard.writeText(code).then(() => {
235
+ const btnSpan = document.querySelector('#copyBtn span');
236
+ btnSpan.textContent = 'Copied!';
237
+ setTimeout(() => { btnSpan.textContent = 'Copy Code'; }, 2000);
238
+ });
239
+ });
240
+ </script>
241
+ </body>
242
+ </html>