Hugging Face's logo

argo11
/
parc2026-openvla-oft-plus-base

Robotics
Transformers
Safetensors
openvla
feature-extraction
custom_code
Model card Files
xet
 Community
argo11 Sylvest commited on
Commit
0d923bf
·
0 Parent(s):

Duplicate from Sylvest/openvla-7b-oft-finetuned-libero-plus-mixdata

Browse files

Co-authored-by: FSY(SII) <Sylvest@users.noreply.huggingface.co>

Files changed (25) hide show
  1. .gitattributes +6 -0
  2. README.md +102 -0
  3. action_head--150000_checkpoint.pt +3 -0
  4. added_tokens.json +3 -0
  5. config.json +3693 -0
  6. configuration_prismatic.py +140 -0
  7. dataset_statistics.json +526 -0
  8. generation_config.json +7 -0
  9. lora_adapter/README.md +202 -0
  10. lora_adapter/adapter_config.json +45 -0
  11. lora_adapter/adapter_model.safetensors +3 -0
  12. model-00001-of-00004.safetensors +3 -0
  13. model-00002-of-00004.safetensors +3 -0
  14. model-00003-of-00004.safetensors +3 -0
  15. model-00004-of-00004.safetensors +3 -0
  16. model.safetensors.index.json +989 -0
  17. modeling_prismatic.py +1085 -0
  18. preprocessor_config.json +114 -0
  19. processing_prismatic.py +257 -0
  20. processor_config.json +6 -0
  21. proprio_projector--150000_checkpoint.pt +3 -0
  22. special_tokens_map.json +30 -0
  23. tokenizer.json +0 -0
  24. tokenizer.model +3 -0
  25. tokenizer_config.json +53 -0
.gitattributes ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ *.glb filter=lfs diff=lfs merge=lfs -text
2
+ model-00004-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text
3
+ lora_adapter/adapter_model.safetensors filter=lfs diff=lfs merge=lfs -text
4
+ action_head--150000_checkpoint.pt filter=lfs diff=lfs merge=lfs -text
5
+ proprio_projector--150000_checkpoint.pt filter=lfs diff=lfs merge=lfs -text
6
+ model-00001-of-00004.safetensors filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ pipeline_tag: robotics
4
+ library_name: transformers
5
+ ---
6
+
7
+ <h1 align="center">
8
+ LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models
9
+ </h1>
10
+
11
+ <p align="center">
12
+ 📄 <a href="https://arxiv.org/pdf/2510.13626v1"><strong>Paper</strong></a> |
13
+ 🏗️ <a href="https://github.com/sylvestf/LIBERO-plus"><strong>Repo</strong></a> |
14
+ 🌐 <a href="https://sylvestf.github.io/LIBERO-plus"><strong>Website</strong></a>
15
+ </p>
16
+
17
+ ## 🔥 Overview
18
+ This repository contains the official implementation and benchmark for our paper "In-depth Robustness Analysis for Vision-Language-Action Models". We systematically expose the hidden vulnerabilities of contemporary VLA models through comprehensive robustness evaluation across seven perturbation dimensions. You can simply replace the original `libero` with a `pip install -e .` without modifying your code.
19
+
20
+ ## 🚀 Key Findings
21
+ - **Significant Fragility**: VLA models exhibit extreme sensitivity to camera viewpoints and robot initial states, with performance dropping from 95% to below 30% under modest perturbations
22
+ - **Language Ignorance**: Models largely ignore language instructions, functioning more like Vision-Action models
23
+ - **Negative Compositional Generalization**: Combined perturbations reveal complex interaction effects beyond independent factors
24
+
25
+ ## 📊 LIBERO-plus Benchmark
26
+
27
+ ### 7 Perturbation Dimensions
28
+ We introduce **LIBERO-plus**, a comprehensive benchmark with 10,030 tasks spanning:
29
+
30
+ 1. **Objects Layout** - Confounding objects and target object displacement
31
+ 2. **Camera Viewpoints** - Position, orientation, and field-of-view changes
32
+ 3. **Robot Initial States** - Manipulator initial pose variations
33
+ 4. **Language Instructions** - LLM-based instruction rewriting
34
+ 5. **Light Conditions** - Intensity, direction, color, and shadow variations
35
+ 6. **Background Textures** - Scene and surface appearance changes
36
+ 7. **Sensor Noise** - Photometric distortions and image degradation
37
+
38
+ ### Evaluated Models
39
+ - OpenVLA and variants (OFT, OFT_w, OFT_m)
40
+ - π₀ and π₀-fast
41
+ - Nora, WorldVLA, UniVLA, RIPT-VLA
42
+
43
+ ## 🛠️ Installation
44
+ Please refer to our [github repo](https://github.com/sylvestf/LIBERO-plus) for more installation details.
45
+ You can download our OpenVLA-OFT weights after mix-SFT from this hf repo. You can also find the [assets](https://huggingface.co/datasets/Sylvest/LIBERO-plus) and the [training dataset](https://huggingface.co/Sylvest/openvla-7b-oft-finetuned-libero-plus-mixdata/tree/main).
46
+
47
+ The extracted directory structure should look like:
48
+
49
+ ```text
50
+ LIBERO-plus/
51
+ └── libero/
52
+ └── libero/
53
+ └── assets/
54
+ ├── articulated_objects/
55
+ ├── new_objects/
56
+ ├── scenes/
57
+ ├── stable_hope_objects/
58
+ ├── stable_scanned_objects/
59
+ ├── textures/
60
+ ├── turbosquid_objects/
61
+ ├── serving_region.xml
62
+ ├── wall_frames.stl
63
+ └── wall.xml
64
+ ```
65
+
66
+ ## 🔧 Evaluation
67
+ The evaluation method is almost identical to `LIBERO`. The only required modification is adjusting `num_trials_per_task` from 50 to 1 in your configuration.
68
+
69
+ ## 📊 LIBERO-Plus Benchmark Leaderboard
70
+ | Model | Camera | Robot | Language | Light | Background | Noise | Layout | Total |
71
+ |-------|--------|-------|----------|-------|------------|-------|--------|-------|
72
+ | [OpenVLA](https://github.com/openvla/openvla) | 0.8 | 3.5 | 23.0 | 8.1 | 50.4 | 15.2 | 28.5 | 17.3 |
73
+ | [OpenVLA-OFT](https://github.com/moojink/openvla-oft) | 56.4 | 31.9 | 79.5 | 88.7 | 97.3 | 75.8 | 74.2 | 70.0 |
74
+ | [OpenVLA-OFT_w](https://github.com/moojink/openvla-oft) | 10.4 | 38.7 | 70.5 | 76.8 | 99.2 | 49.9 | 69.9 | 56.4 |
75
+ | [NORA](https://github.com/declare-lab/nora) | 2.2 | 37.0 | 65.1 | 45.7 | 65.5 | 12.8 | 62.1 | 39.8 |
76
+ | [WorldVLA](https://github.com/alibaba-damo-academy/WorldVLA) | 0.1 | 27.9 | 41.6 | 43.7 | 19.8 | 10.9 | 38.0 | 25.3 |
77
+ | [UniVLA](https://github.com/OpenDriveLab/UniVLA) | 1.8 | 46.2 | 69.6 | 69.0 | 90.7 | 21.2 | 31.9 | 43.9 |
78
+ | [π₀](https://github.com/Physical-Intelligence/openpi) | 13.8 | 6.0 | 58.8 | 85.0 | 90.7 | 79.0 | 68.9 | 54.6 |
79
+ | [π₀-Fast](https://github.com/Physical-Intelligence/openpi) | 65.1 | 21.6 | 61.0 | 73.2 | 97.7 | 74.4 | 68.8 | 64.2 |
80
+ | [RIPT-VLA](https://github.com/Ariostgx/ript-vla) | 55.2 | 31.2 | 77.6 | 88.4 | **100.0** | 73.5 | 74.2 | 69.3 |
81
+ | [OpenVLA-OFT_m](https://github.com/moojink/openvla-oft) | 55.6 | 21.7 | 81.0 | 92.7 | 92.3 | 78.6 | 68.7 | 68.1 |
82
+ | **[OpenVLA-OFT+ (Ours)](https://github.com/moojink/openvla-oft)** | **92.8** | **30.3** | **85.8** | **94.9** | 93.9 | **89.3** | **77.6** | **79.6** |
83
+
84
+ - **OpenVLA-OFT+** shows the performance of [OpenVLA-OFT with a mix-sft on LIBERO-plus dataset](https://huggingface.co/Sylvest/openvla-7b-oft-finetuned-libero-plus-mixdata/tree/main).
85
+ - **OpenVLA-OFT_w** shows the performance of [OpenVLA-OFT without wrist observation input](https://huggingface.co/Sylvest/openvla-7b-oft-finetuned-libero-without-wrist).
86
+ - **OpenVLA-OFT_m** shows the performance of [OpenVLA-OFT with a mix-sft](https://huggingface.co/moojink/openvla-7b-oft-finetuned-libero-spatial).
87
+
88
+ ### Origin LIBERO Benchmark Leaderboard
89
+
90
+ To make it easier to get all the results in one place, we've compiled the evaluation results of current VLA models on the original LIBERO benchmark in this [table](./libero_res.md).
91
+
92
+
93
+ ## Citation
94
+ If you find this work useful for your research, please cite our paper:
95
+ ```bibtex
96
+ @article{fei25libero-plus,
97
+ title={LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models},
98
+ author={Senyu Fei and Siyin Wang and Junhao Shi and Zihao Dai and Jikun Cai and Pengfang Qian and Li Ji and Xinzhe He and Shiduo Zhang and Zhaoye Fei and Jinlan Fu and Jingjing Gong and Xipeng Qiu},
99
+ journal = {arXiv preprint arXiv:2510.13626},
100
+ year={2025},
101
+ }
102
+ ```
action_head--150000_checkpoint.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:adc70a2da8cadd90b5f5cdec1771058130e62e48ede4c8cd351a8c35ee740330
3
+ size 302242674
added_tokens.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "<PAD>": 32000
3
+ }
config.json ADDED
@@ -0,0 +1,3693 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "moojink/openvla-7b-oft-finetuned-libero-spatial-object-goal-10",
3
+ "arch_specifier": "no-align+fused-gelu-mlp",
4
+ "architectures": [
5
+ "OpenVLAForActionPrediction"
6
+ ],
7
+ "auto_map": {
8
+ "AutoConfig": "configuration_prismatic.OpenVLAConfig",
9
+ "AutoModelForVision2Seq": "modeling_prismatic.OpenVLAForActionPrediction"
10
+ },
11
+ "hf_llm_id": "meta-llama/Llama-2-7b-hf",
12
+ "image_resize_strategy": "resize-naive",
13
+ "image_sizes": [
14
+ 224,
15
+ 224
16
+ ],
17
+ "llm_backbone_id": "llama2-7b-pure",
18
+ "llm_max_length": 2048,
19
+ "model_type": "openvla",
20
+ "n_action_bins": 256,
21
+ "norm_stats": {
22
+ "libero_10": {
23
+ "action": {
24
+ "mean": [
25
+ 0.06630383431911469,
26
+ 0.090694859623909,
27
+ -0.09185197949409485,
28
+ 0.00014266508514992893,
29
+ 0.005637437105178833,
30
+ -0.0038363179191946983,
31
+ 0.5176114439964294
32
+ ],
33
+ "std": [
34
+ 0.33593615889549255,
35
+ 0.3820026218891144,
36
+ 0.4535250663757324,
37
+ 0.03832726553082466,
38
+ 0.06335575133562088,
39
+ 0.07599934190511703,
40
+ 0.4993402063846588
41
+ ],
42
+ "max": [
43
+ 0.9375,
44
+ 0.9375,
45
+ 0.9375,
46
+ 0.3557142913341522,
47
+ 0.375,
48
+ 0.375,
49
+ 1.0
50
+ ],
51
+ "min": [
52
+ -0.9375,
53
+ -0.9375,
54
+ -0.9375,
55
+ -0.2582142949104309,
56
+ -0.375,
57
+ -0.3642857074737549,
58
+ 0.0
59
+ ],
60
+ "q01": [
61
+ -0.6883928775787354,
62
+ -0.8089285492897034,
63
+ -0.9375,
64
+ -0.1128571406006813,
65
+ -0.1639285683631897,
66
+ -0.20357142388820648,
67
+ 0.0
68
+ ],
69
+ "q99": [
70
+ 0.9375,
71
+ 0.8705357313156128,
72
+ 0.9375,
73
+ 0.12428571283817291,
74
+ 0.19285714626312256,
75
+ 0.3449999988079071,
76
+ 1.0
77
+ ],
78
+ "mask": [
79
+ true,
80
+ true,
81
+ true,
82
+ true,
83
+ true,
84
+ true,
85
+ false
86
+ ]
87
+ },
88
+ "proprio": {
89
+ "mean": [
90
+ -0.04460641369223595,
91
+ 0.035552240908145905,
92
+ 0.7428034543991089,
93
+ 2.9607746601104736,
94
+ -0.13809093832969666,
95
+ -0.10721249878406525,
96
+ 0.02668767236173153,
97
+ -0.02694464474916458
98
+ ],
99
+ "std": [
100
+ 0.10515860468149185,
101
+ 0.1530338078737259,
102
+ 0.38418349623680115,
103
+ 0.3309749960899353,
104
+ 0.8265467882156372,
105
+ 0.311494916677475,
106
+ 0.014123727567493916,
107
+ 0.014056944288313389
108
+ ],
109
+ "max": [
110
+ 0.21031762659549713,
111
+ 0.39128610491752625,
112
+ 1.3660105466842651,
113
+ 3.7743608951568604,
114
+ 3.5752453804016113,
115
+ 1.3747878074645996,
116
+ 0.042380549013614655,
117
+ 0.0010137537028640509
118
+ ],
119
+ "min": [
120
+ -0.4856736361980438,
121
+ -0.3313629925251007,
122
+ 0.008128180168569088,
123
+ 0.8415665030479431,
124
+ -3.616978406906128,
125
+ -1.842738389968872,
126
+ -0.002542249858379364,
127
+ -0.042119648307561874
128
+ ],
129
+ "q01": [
130
+ -0.40087422728538513,
131
+ -0.2639563336968422,
132
+ 0.029872100800275803,
133
+ 1.5078086853027344,
134
+ -2.591851234436035,
135
+ -1.0244591236114502,
136
+ 0.0017080969410017133,
137
+ -0.04002029821276665
138
+ ],
139
+ "q99": [
140
+ 0.13566908240318298,
141
+ 0.335424880683422,
142
+ 1.2716039419174194,
143
+ 3.2693305015563965,
144
+ 2.4190170764923096,
145
+ 0.6207390427589417,
146
+ 0.04029817134141922,
147
+ -0.0017327156383544207
148
+ ]
149
+ },
150
+ "num_transitions": 2238036,
151
+ "num_trajectories": 14347
152
+ },
153
+ "libero_object": {
154
+ "action": {
155
+ "mean": [
156
+ 0.06630383431911469,
157
+ 0.090694859623909,
158
+ -0.09185197949409485,
159
+ 0.00014266508514992893,
160
+ 0.005637437105178833,
161
+ -0.0038363179191946983,
162
+ 0.5176114439964294
163
+ ],
164
+ "std": [
165
+ 0.33593615889549255,
166
+ 0.3820026218891144,
167
+ 0.4535250663757324,
168
+ 0.03832726553082466,
169
+ 0.06335575133562088,
170
+ 0.07599934190511703,
171
+ 0.4993402063846588
172
+ ],
173
+ "max": [
174
+ 0.9375,
175
+ 0.9375,
176
+ 0.9375,
177
+ 0.3557142913341522,
178
+ 0.375,
179
+ 0.375,
180
+ 1.0
181
+ ],
182
+ "min": [
183
+ -0.9375,
184
+ -0.9375,
185
+ -0.9375,
186
+ -0.2582142949104309,
187
+ -0.375,
188
+ -0.3642857074737549,
189
+ 0.0
190
+ ],
191
+ "q01": [
192
+ -0.6883928775787354,
193
+ -0.8089285492897034,
194
+ -0.9375,
195
+ -0.1128571406006813,
196
+ -0.1639285683631897,
197
+ -0.20357142388820648,
198
+ 0.0
199
+ ],
200
+ "q99": [
201
+ 0.9375,
202
+ 0.8705357313156128,
203
+ 0.9375,
204
+ 0.12428571283817291,
205
+ 0.19285714626312256,
206
+ 0.3449999988079071,
207
+ 1.0
208
+ ],
209
+ "mask": [
210
+ true,
211
+ true,
212
+ true,
213
+ true,
214
+ true,
215
+ true,
216
+ false
217
+ ]
218
+ },
219
+ "proprio": {
220
+ "mean": [
221
+ -0.04460641369223595,
222
+ 0.035552240908145905,
223
+ 0.7428034543991089,
224
+ 2.9607746601104736,
225
+ -0.13809093832969666,
226
+ -0.10721249878406525,
227
+ 0.02668767236173153,
228
+ -0.02694464474916458
229
+ ],
230
+ "std": [
231
+ 0.10515860468149185,
232
+ 0.1530338078737259,
233
+ 0.38418349623680115,
234
+ 0.3309749960899353,
235
+ 0.8265467882156372,
236
+ 0.311494916677475,
237
+ 0.014123727567493916,
238
+ 0.014056944288313389
239
+ ],
240
+ "max": [
241
+ 0.21031762659549713,
242
+ 0.39128610491752625,
243
+ 1.3660105466842651,
244
+ 3.7743608951568604,
245
+ 3.5752453804016113,
246
+ 1.3747878074645996,
247
+ 0.042380549013614655,
248
+ 0.0010137537028640509
249
+ ],
250
+ "min": [
251
+ -0.4856736361980438,
252
+ -0.3313629925251007,
253
+ 0.008128180168569088,
254
+ 0.8415665030479431,
255
+ -3.616978406906128,
256
+ -1.842738389968872,
257
+ -0.002542249858379364,
258
+ -0.042119648307561874
259
+ ],
260
+ "q01": [
261
+ -0.40087422728538513,
262
+ -0.2639563336968422,
263
+ 0.029872100800275803,
264
+ 1.5078086853027344,
265
+ -2.591851234436035,
266
+ -1.0244591236114502,
267
+ 0.0017080969410017133,
268
+ -0.04002029821276665
269
+ ],
270
+ "q99": [
271
+ 0.13566908240318298,
272
+ 0.335424880683422,
273
+ 1.2716039419174194,
274
+ 3.2693305015563965,
275
+ 2.4190170764923096,
276
+ 0.6207390427589417,
277
+ 0.04029817134141922,
278
+ -0.0017327156383544207
279
+ ]
280
+ },
281
+ "num_transitions": 2238036,
282
+ "num_trajectories": 14347
283
+ },
284
+ "libero_goal": {
285
+ "action": {
286
+ "mean": [
287
+ 0.06630383431911469,
288
+ 0.090694859623909,
289
+ -0.09185197949409485,
290
+ 0.00014266508514992893,
291
+ 0.005637437105178833,
292
+ -0.0038363179191946983,
293
+ 0.5176114439964294
294
+ ],
295
+ "std": [
296
+ 0.33593615889549255,
297
+ 0.3820026218891144,
298
+ 0.4535250663757324,
299
+ 0.03832726553082466,
300
+ 0.06335575133562088,
301
+ 0.07599934190511703,
302
+ 0.4993402063846588
303
+ ],
304
+ "max": [
305
+ 0.9375,
306
+ 0.9375,
307
+ 0.9375,
308
+ 0.3557142913341522,
309
+ 0.375,
310
+ 0.375,
311
+ 1.0
312
+ ],
313
+ "min": [
314
+ -0.9375,
315
+ -0.9375,
316
+ -0.9375,
317
+ -0.2582142949104309,
318
+ -0.375,
319
+ -0.3642857074737549,
320
+ 0.0
321
+ ],
322
+ "q01": [
323
+ -0.6883928775787354,
324
+ -0.8089285492897034,
325
+ -0.9375,
326
+ -0.1128571406006813,
327
+ -0.1639285683631897,
328
+ -0.20357142388820648,
329
+ 0.0
330
+ ],
331
+ "q99": [
332
+ 0.9375,
333
+ 0.8705357313156128,
334
+ 0.9375,
335
+ 0.12428571283817291,
336
+ 0.19285714626312256,
337
+ 0.3449999988079071,
338
+ 1.0
339
+ ],
340
+ "mask": [
341
+ true,
342
+ true,
343
+ true,
344
+ true,
345
+ true,
346
+ true,
347
+ false
348
+ ]
349
+ },
350
+ "proprio": {
351
+ "mean": [
352
+ -0.04460641369223595,
353
+ 0.035552240908145905,
354
+ 0.7428034543991089,
355
+ 2.9607746601104736,
356
+ -0.13809093832969666,
357
+ -0.10721249878406525,
358
+ 0.02668767236173153,
359
+ -0.02694464474916458
360
+ ],
361
+ "std": [
362
+ 0.10515860468149185,
363
+ 0.1530338078737259,
364
+ 0.38418349623680115,
365
+ 0.3309749960899353,
366
+ 0.8265467882156372,
367
+ 0.311494916677475,
368
+ 0.014123727567493916,
369
+ 0.014056944288313389
370
+ ],
371
+ "max": [
372
+ 0.21031762659549713,
373
+ 0.39128610491752625,
374
+ 1.3660105466842651,
375
+ 3.7743608951568604,
376
+ 3.5752453804016113,
377
+ 1.3747878074645996,
378
+ 0.042380549013614655,
379
+ 0.0010137537028640509
380
+ ],
381
+ "min": [
382
+ -0.4856736361980438,
383
+ -0.3313629925251007,
384
+ 0.008128180168569088,
385
+ 0.8415665030479431,
386
+ -3.616978406906128,
387
+ -1.842738389968872,
388
+ -0.002542249858379364,
389
+ -0.042119648307561874
390
+ ],
391
+ "q01": [
392
+ -0.40087422728538513,
393
+ -0.2639563336968422,
394
+ 0.029872100800275803,
395
+ 1.5078086853027344,
396
+ -2.591851234436035,
397
+ -1.0244591236114502,
398
+ 0.0017080969410017133,
399
+ -0.04002029821276665
400
+ ],
401
+ "q99": [
402
+ 0.13566908240318298,
403
+ 0.335424880683422,
404
+ 1.2716039419174194,
405
+ 3.2693305015563965,
406
+ 2.4190170764923096,
407
+ 0.6207390427589417,
408
+ 0.04029817134141922,
409
+ -0.0017327156383544207
410
+ ]
411
+ },
412
+ "num_transitions": 2238036,
413
+ "num_trajectories": 14347
414
+ },
415
+ "libero_spatial": {
416
+ "action": {
417
+ "mean": [
418
+ 0.06630383431911469,
419
+ 0.090694859623909,
420
+ -0.09185197949409485,
421
+ 0.00014266508514992893,
422
+ 0.005637437105178833,
423
+ -0.0038363179191946983,
424
+ 0.5176114439964294
425
+ ],
426
+ "std": [
427
+ 0.33593615889549255,
428
+ 0.3820026218891144,
429
+ 0.4535250663757324,
430
+ 0.03832726553082466,
431
+ 0.06335575133562088,
432
+ 0.07599934190511703,
433
+ 0.4993402063846588
434
+ ],
435
+ "max": [
436
+ 0.9375,
437
+ 0.9375,
438
+ 0.9375,
439
+ 0.3557142913341522,
440
+ 0.375,
441
+ 0.375,
442
+ 1.0
443
+ ],
444
+ "min": [
445
+ -0.9375,
446
+ -0.9375,
447
+ -0.9375,
448
+ -0.2582142949104309,
449
+ -0.375,
450
+ -0.3642857074737549,
451
+ 0.0
452
+ ],
453
+ "q01": [
454
+ -0.6883928775787354,
455
+ -0.8089285492897034,
456
+ -0.9375,
457
+ -0.1128571406006813,
458
+ -0.1639285683631897,
459
+ -0.20357142388820648,
460
+ 0.0
461
+ ],
462
+ "q99": [
463
+ 0.9375,
464
+ 0.8705357313156128,
465
+ 0.9375,
466
+ 0.12428571283817291,
467
+ 0.19285714626312256,
468
+ 0.3449999988079071,
469
+ 1.0
470
+ ],
471
+ "mask": [
472
+ true,
473
+ true,
474
+ true,
475
+ true,
476
+ true,
477
+ true,
478
+ false
479
+ ]
480
+ },
481
+ "proprio": {
482
+ "mean": [
483
+ -0.04460641369223595,
484
+ 0.035552240908145905,
485
+ 0.7428034543991089,
486
+ 2.9607746601104736,
487
+ -0.13809093832969666,
488
+ -0.10721249878406525,
489
+ 0.02668767236173153,
490
+ -0.02694464474916458
491
+ ],
492
+ "std": [
493
+ 0.10515860468149185,
494
+ 0.1530338078737259,
495
+ 0.38418349623680115,
496
+ 0.3309749960899353,
497
+ 0.8265467882156372,
498
+ 0.311494916677475,
499
+ 0.014123727567493916,
500
+ 0.014056944288313389
501
+ ],
502
+ "max": [
503
+ 0.21031762659549713,
504
+ 0.39128610491752625,
505
+ 1.3660105466842651,
506
+ 3.7743608951568604,
507
+ 3.5752453804016113,
508
+ 1.3747878074645996,
509
+ 0.042380549013614655,
510
+ 0.0010137537028640509
511
+ ],
512
+ "min": [
513
+ -0.4856736361980438,
514
+ -0.3313629925251007,
515
+ 0.008128180168569088,
516
+ 0.8415665030479431,
517
+ -3.616978406906128,
518
+ -1.842738389968872,
519
+ -0.002542249858379364,
520
+ -0.042119648307561874
521
+ ],
522
+ "q01": [
523
+ -0.40087422728538513,
524
+ -0.2639563336968422,
525
+ 0.029872100800275803,
526
+ 1.5078086853027344,
527
+ -2.591851234436035,
528
+ -1.0244591236114502,
529
+ 0.0017080969410017133,
530
+ -0.04002029821276665
531
+ ],
532
+ "q99": [
533
+ 0.13566908240318298,
534
+ 0.335424880683422,
535
+ 1.2716039419174194,
536
+ 3.2693305015563965,
537
+ 2.4190170764923096,
538
+ 0.6207390427589417,
539
+ 0.04029817134141922,
540
+ -0.0017327156383544207
541
+ ]
542
+ },
543
+ "num_transitions": 2238036,
544
+ "num_trajectories": 14347
545
+ },
546
+ "austin_buds_dataset_converted_externally_to_rlds": {
547
+ "action": {
548
+ "mask": [
549
+ true,
550
+ true,
551
+ true,
552
+ true,
553
+ true,
554
+ true,
555
+ false
556
+ ],
557
+ "max": [
558
+ 1.0,
559
+ 1.0,
560
+ 1.0,
561
+ 0.0,
562
+ 0.0,
563
+ 0.0,
564
+ 1.0
565
+ ],
566
+ "mean": [
567
+ -0.07678354531526566,
568
+ 0.0036849044263362885,
569
+ 0.05644911900162697,
570
+ 0.0,
571
+ 0.0,
572
+ 0.0,
573
+ 0.3510494828224182
574
+ ],
575
+ "min": [
576
+ -1.0,
577
+ -1.0,
578
+ -1.0,
579
+ 0.0,
580
+ 0.0,
581
+ 0.0,
582
+ 0.0
583
+ ],
584
+ "q01": [
585
+ -1.0,
586
+ -0.9599999785423279,
587
+ -0.8714285492897034,
588
+ 0.0,
589
+ 0.0,
590
+ 0.0,
591
+ 0.0
592
+ ],
593
+ "q99": [
594
+ 1.0,
595
+ 0.8600000143051147,
596
+ 1.0,
597
+ 0.0,
598
+ 0.0,
599
+ 0.0,
600
+ 1.0
601
+ ],
602
+ "std": [
603
+ 0.6367740631103516,
604
+ 0.37889179587364197,
605
+ 0.47796326875686646,
606
+ 0.0,
607
+ 0.0,
608
+ 0.0,
609
+ 0.47721168398857117
610
+ ]
611
+ },
612
+ "num_trajectories": 50,
613
+ "num_transitions": 34112,
614
+ "proprio": {
615
+ "max": [
616
+ 0.0,
617
+ 0.0,
618
+ 0.0,
619
+ 0.0,
620
+ 0.0,
621
+ 0.0,
622
+ 0.0
623
+ ],
624
+ "mean": [
625
+ 0.0,
626
+ 0.0,
627
+ 0.0,
628
+ 0.0,
629
+ 0.0,
630
+ 0.0,
631
+ 0.0
632
+ ],
633
+ "min": [
634
+ 0.0,
635
+ 0.0,
636
+ 0.0,
637
+ 0.0,
638
+ 0.0,
639
+ 0.0,
640
+ 0.0
641
+ ],
642
+ "q01": [
643
+ 0.0,
644
+ 0.0,
645
+ 0.0,
646
+ 0.0,
647
+ 0.0,
648
+ 0.0,
649
+ 0.0
650
+ ],
651
+ "q99": [
652
+ 0.0,
653
+ 0.0,
654
+ 0.0,
655
+ 0.0,
656
+ 0.0,
657
+ 0.0,
658
+ 0.0
659
+ ],
660
+ "std": [
661
+ 0.0,
662
+ 0.0,
663
+ 0.0,
664
+ 0.0,
665
+ 0.0,
666
+ 0.0,
667
+ 0.0
668
+ ]
669
+ }
670
+ },
671
+ "austin_sailor_dataset_converted_externally_to_rlds": {
672
+ "action": {
673
+ "mask": [
674
+ true,
675
+ true,
676
+ true,
677
+ true,
678
+ true,
679
+ true,
680
+ false
681
+ ],
682
+ "max": [
683
+ 1.0,
684
+ 1.0,
685
+ 1.0,
686
+ 0.0,
687
+ 0.0,
688
+ 0.375,
689
+ 1.0
690
+ ],
691
+ "mean": [
692
+ 0.011825348250567913,
693
+ 0.006461074110120535,
694
+ 0.06023626774549484,
695
+ 0.0,
696
+ 0.0,
697
+ 0.0016465914668515325,
698
+ 0.5260950326919556
699
+ ],
700
+ "min": [
701
+ -1.0,
702
+ -1.0,
703
+ -1.0,
704
+ 0.0,
705
+ 0.0,
706
+ -0.375,
707
+ 0.0
708
+ ],
709
+ "q01": [
710
+ -1.0,
711
+ -0.9828571677207947,
712
+ -0.6000000238418579,
713
+ 0.0,
714
+ 0.0,
715
+ -0.17249999940395355,
716
+ 0.0
717
+ ],
718
+ "q99": [
719
+ 1.0,
720
+ 0.9457142949104309,
721
+ 1.0,
722
+ 0.0,
723
+ 0.0,
724
+ 0.17892856895923615,
725
+ 1.0
726
+ ],
727
+ "std": [
728
+ 0.46348899602890015,
729
+ 0.41240179538726807,
730
+ 0.411862850189209,
731
+ 0.0,
732
+ 0.0,
733
+ 0.0578610822558403,
734
+ 0.49894046783447266
735
+ ]
736
+ },
737
+ "num_trajectories": 240,
738
+ "num_transitions": 353094,
739
+ "proprio": {
740
+ "max": [
741
+ 0.0,
742
+ 0.0,
743
+ 0.0,
744
+ 0.0,
745
+ 0.0,
746
+ 0.0,
747
+ 0.0
748
+ ],
749
+ "mean": [
750
+ 0.0,
751
+ 0.0,
752
+ 0.0,
753
+ 0.0,
754
+ 0.0,
755
+ 0.0,
756
+ 0.0
757
+ ],
758
+ "min": [
759
+ 0.0,
760
+ 0.0,
761
+ 0.0,
762
+ 0.0,
763
+ 0.0,
764
+ 0.0,
765
+ 0.0
766
+ ],
767
+ "q01": [
768
+ 0.0,
769
+ 0.0,
770
+ 0.0,
771
+ 0.0,
772
+ 0.0,
773
+ 0.0,
774
+ 0.0
775
+ ],
776
+ "q99": [
777
+ 0.0,
778
+ 0.0,
779
+ 0.0,
780
+ 0.0,
781
+ 0.0,
782
+ 0.0,
783
+ 0.0
784
+ ],
785
+ "std": [
786
+ 0.0,
787
+ 0.0,
788
+ 0.0,
789
+ 0.0,
790
+ 0.0,
791
+ 0.0,
792
+ 0.0
793
+ ]
794
+ }
795
+ },
796
+ "austin_sirius_dataset_converted_externally_to_rlds": {
797
+ "action": {
798
+ "mask": [
799
+ true,
800
+ true,
801
+ true,
802
+ true,
803
+ true,
804
+ true,
805
+ false
806
+ ],
807
+ "max": [
808
+ 1.0002285242080688,
809
+ 0.960608720779419,
810
+ 1.105179786682129,
811
+ 0.0,
812
+ 0.0,
813
+ 0.341785728931427,
814
+ 1.0
815
+ ],
816
+ "mean": [
817
+ 0.07747682929039001,
818
+ 0.03195561468601227,
819
+ 0.04244732856750488,
820
+ 0.0,
821
+ 0.0,
822
+ -0.01603456400334835,
823
+ 0.43260177969932556
824
+ ],
825
+ "min": [
826
+ -1.0183025598526,
827
+ -0.9800000190734863,
828
+ -0.9774575233459473,
829
+ 0.0,
830
+ 0.0,
831
+ -0.34607142210006714,
832
+ 0.0
833
+ ],
834
+ "q01": [
835
+ -0.780905865430832,
836
+ -0.5667179036140442,
837
+ -0.5254343223571777,
838
+ 0.0,
839
+ 0.0,
840
+ -0.28495091378688814,
841
+ 0.0
842
+ ],
843
+ "q99": [
844
+ 0.9569637751579284,
845
+ 0.6971374487876891,
846
+ 0.8124888157844541,
847
+ 0.0,
848
+ 0.0,
849
+ 0.1971428543329239,
850
+ 1.0
851
+ ],
852
+ "std": [
853
+ 0.3906329572200775,
854
+ 0.2998155355453491,
855
+ 0.2782271206378937,
856
+ 0.0,
857
+ 0.0,
858
+ 0.08120622485876083,
859
+ 0.49528297781944275
860
+ ]
861
+ },
862
+ "num_trajectories": 559,
863
+ "num_transitions": 279939,
864
+ "proprio": {
865
+ "max": [
866
+ 0.0,
867
+ 0.0,
868
+ 0.0,
869
+ 0.0,
870
+ 0.0,
871
+ 0.0,
872
+ 0.0
873
+ ],
874
+ "mean": [
875
+ 0.0,
876
+ 0.0,
877
+ 0.0,
878
+ 0.0,
879
+ 0.0,
880
+ 0.0,
881
+ 0.0
882
+ ],
883
+ "min": [
884
+ 0.0,
885
+ 0.0,
886
+ 0.0,
887
+ 0.0,
888
+ 0.0,
889
+ 0.0,
890
+ 0.0
891
+ ],
892
+ "q01": [
893
+ 0.0,
894
+ 0.0,
895
+ 0.0,
896
+ 0.0,
897
+ 0.0,
898
+ 0.0,
899
+ 0.0
900
+ ],
901
+ "q99": [
902
+ 0.0,
903
+ 0.0,
904
+ 0.0,
905
+ 0.0,
906
+ 0.0,
907
+ 0.0,
908
+ 0.0
909
+ ],
910
+ "std": [
911
+ 0.0,
912
+ 0.0,
913
+ 0.0,
914
+ 0.0,
915
+ 0.0,
916
+ 0.0,
917
+ 0.0
918
+ ]
919
+ }
920
+ },
921
+ "bc_z": {
922
+ "action": {
923
+ "mask": [
924
+ true,
925
+ true,
926
+ true,
927
+ true,
928
+ true,
929
+ true,
930
+ false
931
+ ],
932
+ "max": [
933
+ 0.2165454924106598,
934
+ 0.1251407265663147,
935
+ 0.10772687941789627,
936
+ 0.33544227480888367,
937
+ 0.28117990493774414,
938
+ 0.40614867210388184,
939
+ 1.0
940
+ ],
941
+ "mean": [
942
+ -0.009958467446267605,
943
+ 0.0008958321413956583,
944
+ 0.004995597992092371,
945
+ 0.00029755113064311445,
946
+ -0.008735382929444313,
947
+ -0.030693737789988518,
948
+ 0.8344562649726868
949
+ ],
950
+ "min": [
951
+ -0.1677047461271286,
952
+ -0.14630407094955444,
953
+ -0.10066790133714676,
954
+ -0.29421567916870117,
955
+ -0.32101404666900635,
956
+ -0.4635624885559082,
957
+ 0.0
958
+ ],
959
+ "q01": [
960
+ -0.09220654994249344,
961
+ -0.06456145539879798,
962
+ -0.049121275544166565,
963
+ -0.11594625547528267,
964
+ -0.14152548640966414,
965
+ -0.2251061636209488,
966
+ 0.0
967
+ ],
968
+ "q99": [
969
+ 0.07628866866230968,
970
+ 0.058019736707210584,
971
+ 0.052540797740221024,
972
+ 0.11740604028105736,
973
+ 0.11703975558280955,
974
+ 0.16729306846857078,
975
+ 1.0
976
+ ],
977
+ "std": [
978
+ 0.03053455986082554,
979
+ 0.0231423731893301,
980
+ 0.020641816779971123,
981
+ 0.04155943542718887,
982
+ 0.046427831053733826,
983
+ 0.0769818127155304,
984
+ 0.3610210120677948
985
+ ]
986
+ },
987
+ "num_trajectories": 43264,
988
+ "num_transitions": 6015535,
989
+ "proprio": {
990
+ "max": [
991
+ 0.0,
992
+ 0.0,
993
+ 0.0,
994
+ 0.0,
995
+ 0.0,
996
+ 0.0,
997
+ 0.0
998
+ ],
999
+ "mean": [
1000
+ 0.0,
1001
+ 0.0,
1002
+ 0.0,
1003
+ 0.0,
1004
+ 0.0,
1005
+ 0.0,
1006
+ 0.0
1007
+ ],
1008
+ "min": [
1009
+ 0.0,
1010
+ 0.0,
1011
+ 0.0,
1012
+ 0.0,
1013
+ 0.0,
1014
+ 0.0,
1015
+ 0.0
1016
+ ],
1017
+ "q01": [
1018
+ 0.0,
1019
+ 0.0,
1020
+ 0.0,
1021
+ 0.0,
1022
+ 0.0,
1023
+ 0.0,
1024
+ 0.0
1025
+ ],
1026
+ "q99": [
1027
+ 0.0,
1028
+ 0.0,
1029
+ 0.0,
1030
+ 0.0,
1031
+ 0.0,
1032
+ 0.0,
1033
+ 0.0
1034
+ ],
1035
+ "std": [
1036
+ 0.0,
1037
+ 0.0,
1038
+ 0.0,
1039
+ 0.0,
1040
+ 0.0,
1041
+ 0.0,
1042
+ 0.0
1043
+ ]
1044
+ }
1045
+ },
1046
+ "berkeley_autolab_ur5": {
1047
+ "action": {
1048
+ "mask": [
1049
+ true,
1050
+ true,
1051
+ true,
1052
+ true,
1053
+ true,
1054
+ true,
1055
+ false
1056
+ ],
1057
+ "max": [
1058
+ 0.019999999552965164,
1059
+ 0.019999999552965164,
1060
+ 0.019999999552965164,
1061
+ 0.06666667014360428,
1062
+ 0.06666667014360428,
1063
+ 0.06666667014360428,
1064
+ 1.0
1065
+ ],
1066
+ "mean": [
1067
+ 0.0005683620693162084,
1068
+ 0.001217700308188796,
1069
+ -0.0005296372692100704,
1070
+ 0.00021029810886830091,
1071
+ 6.0695128922816366e-05,
1072
+ 0.001204986940138042,
1073
+ 0.6298308372497559
1074
+ ],
1075
+ "min": [
1076
+ -0.019999999552965164,
1077
+ -0.019999999552965164,
1078
+ -0.019999999552965164,
1079
+ -0.06666667014360428,
1080
+ -0.06666667014360428,
1081
+ -0.06666667014360428,
1082
+ 0.0
1083
+ ],
1084
+ "q01": [
1085
+ -0.019999999552965164,
1086
+ -0.019999999552965164,
1087
+ -0.019999999552965164,
1088
+ -0.02628571353852749,
1089
+ -0.06666667014360428,
1090
+ -0.03847619146108627,
1091
+ 0.0
1092
+ ],
1093
+ "q99": [
1094
+ 0.019999999552965164,
1095
+ 0.019999999552965164,
1096
+ 0.019999999552965164,
1097
+ 0.031809523701667786,
1098
+ 0.06666667014360428,
1099
+ 0.036571428179740906,
1100
+ 1.0
1101
+ ],
1102
+ "std": [
1103
+ 0.0115329809486866,
1104
+ 0.007990492507815361,
1105
+ 0.009577835910022259,
1106
+ 0.009432995691895485,
1107
+ 0.016427582129836082,
1108
+ 0.011053967289626598,
1109
+ 0.48267969489097595
1110
+ ]
1111
+ },
1112
+ "num_trajectories": 1000,
1113
+ "num_transitions": 97939,
1114
+ "proprio": {
1115
+ "max": [
1116
+ 0.0,
1117
+ 0.0,
1118
+ 0.0,
1119
+ 0.0,
1120
+ 0.0,
1121
+ 0.0,
1122
+ 0.0
1123
+ ],
1124
+ "mean": [
1125
+ 0.0,
1126
+ 0.0,
1127
+ 0.0,
1128
+ 0.0,
1129
+ 0.0,
1130
+ 0.0,
1131
+ 0.0
1132
+ ],
1133
+ "min": [
1134
+ 0.0,
1135
+ 0.0,
1136
+ 0.0,
1137
+ 0.0,
1138
+ 0.0,
1139
+ 0.0,
1140
+ 0.0
1141
+ ],
1142
+ "q01": [
1143
+ 0.0,
1144
+ 0.0,
1145
+ 0.0,
1146
+ 0.0,
1147
+ 0.0,
1148
+ 0.0,
1149
+ 0.0
1150
+ ],
1151
+ "q99": [
1152
+ 0.0,
1153
+ 0.0,
1154
+ 0.0,
1155
+ 0.0,
1156
+ 0.0,
1157
+ 0.0,
1158
+ 0.0
1159
+ ],
1160
+ "std": [
1161
+ 0.0,
1162
+ 0.0,
1163
+ 0.0,
1164
+ 0.0,
1165
+ 0.0,
1166
+ 0.0,
1167
+ 0.0
1168
+ ]
1169
+ }
1170
+ },
1171
+ "berkeley_cable_routing": {
1172
+ "action": {
1173
+ "mask": [
1174
+ true,
1175
+ true,
1176
+ true,
1177
+ true,
1178
+ true,
1179
+ true,
1180
+ false
1181
+ ],
1182
+ "max": [
1183
+ 0.9633283019065857,
1184
+ 1.0,
1185
+ 1.0,
1186
+ 0.0,
1187
+ 0.0,
1188
+ 1.0,
1189
+ 0.0
1190
+ ],
1191
+ "mean": [
1192
+ -0.07139874249696732,
1193
+ 0.023609008640050888,
1194
+ 0.10241943597793579,
1195
+ 0.0,
1196
+ 0.0,
1197
+ 0.049671024084091187,
1198
+ 0.0
1199
+ ],
1200
+ "min": [
1201
+ -0.9809081554412842,
1202
+ -0.9554349184036255,
1203
+ -0.9994775056838989,
1204
+ 0.0,
1205
+ 0.0,
1206
+ -1.0,
1207
+ 0.0
1208
+ ],
1209
+ "q01": [
1210
+ -0.5534318816661835,
1211
+ -0.4797285574674606,
1212
+ -0.5314934802055359,
1213
+ 0.0,
1214
+ 0.0,
1215
+ -0.8855219376087189,
1216
+ 0.0
1217
+ ],
1218
+ "q99": [
1219
+ 0.42652835428714786,
1220
+ 0.5000944086909298,
1221
+ 0.639823433756829,
1222
+ 0.0,
1223
+ 0.0,
1224
+ 0.984243879914284,
1225
+ 0.0
1226
+ ],
1227
+ "std": [
1228
+ 0.1815500408411026,
1229
+ 0.1810990273952484,
1230
+ 0.21220779418945312,
1231
+ 0.0,
1232
+ 0.0,
1233
+ 0.3475511968135834,
1234
+ 0.0
1235
+ ]
1236
+ },
1237
+ "num_trajectories": 1647,
1238
+ "num_transitions": 42328,
1239
+ "proprio": {
1240
+ "max": [
1241
+ 0.0,
1242
+ 0.0,
1243
+ 0.0,
1244
+ 0.0,
1245
+ 0.0,
1246
+ 0.0,
1247
+ 0.0
1248
+ ],
1249
+ "mean": [
1250
+ 0.0,
1251
+ 0.0,
1252
+ 0.0,
1253
+ 0.0,
1254
+ 0.0,
1255
+ 0.0,
1256
+ 0.0
1257
+ ],
1258
+ "min": [
1259
+ 0.0,
1260
+ 0.0,
1261
+ 0.0,
1262
+ 0.0,
1263
+ 0.0,
1264
+ 0.0,
1265
+ 0.0
1266
+ ],
1267
+ "q01": [
1268
+ 0.0,
1269
+ 0.0,
1270
+ 0.0,
1271
+ 0.0,
1272
+ 0.0,
1273
+ 0.0,
1274
+ 0.0
1275
+ ],
1276
+ "q99": [
1277
+ 0.0,
1278
+ 0.0,
1279
+ 0.0,
1280
+ 0.0,
1281
+ 0.0,
1282
+ 0.0,
1283
+ 0.0
1284
+ ],
1285
+ "std": [
1286
+ 0.0,
1287
+ 0.0,
1288
+ 0.0,
1289
+ 0.0,
1290
+ 0.0,
1291
+ 0.0,
1292
+ 0.0
1293
+ ]
1294
+ }
1295
+ },
1296
+ "berkeley_fanuc_manipulation": {
1297
+ "action": {
1298
+ "mask": [
1299
+ true,
1300
+ true,
1301
+ true,
1302
+ true,
1303
+ true,
1304
+ true,
1305
+ false
1306
+ ],
1307
+ "max": [
1308
+ 0.009999999776482582,
1309
+ 0.009999999776482582,
1310
+ 0.009999999776482582,
1311
+ 0.03490658476948738,
1312
+ 0.03490658476948738,
1313
+ 0.03490658476948738,
1314
+ 1.0
1315
+ ],
1316
+ "mean": [
1317
+ 0.0007744057802483439,
1318
+ -0.00031240080716088414,
1319
+ -0.0015001941937953234,
1320
+ -0.0007515158504247665,
1321
+ -0.00015832878125365824,
1322
+ 0.00014327642566058785,
1323
+ 0.699295699596405
1324
+ ],
1325
+ "min": [
1326
+ -0.009999999776482582,
1327
+ -0.009999999776482582,
1328
+ -0.009999999776482582,
1329
+ -0.03490658476948738,
1330
+ -0.03490658476948738,
1331
+ -0.03490658476948738,
1332
+ 0.0
1333
+ ],
1334
+ "q01": [
1335
+ -0.009999999776482582,
1336
+ -0.009999999776482582,
1337
+ -0.009999999776482582,
1338
+ -0.03490658476948738,
1339
+ 0.0,
1340
+ -0.03490658476948738,
1341
+ 0.0
1342
+ ],
1343
+ "q99": [
1344
+ 0.009999999776482582,
1345
+ 0.009999999776482582,
1346
+ 0.009999999776482582,
1347
+ 0.03490658476948738,
1348
+ 0.0,
1349
+ 0.03490658476948738,
1350
+ 1.0
1351
+ ],
1352
+ "std": [
1353
+ 0.0034070091787725687,
1354
+ 0.0049921851605176926,
1355
+ 0.005344334989786148,
1356
+ 0.00759894959628582,
1357
+ 0.004081866703927517,
1358
+ 0.008568956516683102,
1359
+ 0.4586937427520752
1360
+ ]
1361
+ },
1362
+ "num_trajectories": 415,
1363
+ "num_transitions": 62613,
1364
+ "proprio": {
1365
+ "max": [
1366
+ 0.0,
1367
+ 0.0,
1368
+ 0.0,
1369
+ 0.0,
1370
+ 0.0,
1371
+ 0.0,
1372
+ 0.0
1373
+ ],
1374
+ "mean": [
1375
+ 0.0,
1376
+ 0.0,
1377
+ 0.0,
1378
+ 0.0,
1379
+ 0.0,
1380
+ 0.0,
1381
+ 0.0
1382
+ ],
1383
+ "min": [
1384
+ 0.0,
1385
+ 0.0,
1386
+ 0.0,
1387
+ 0.0,
1388
+ 0.0,
1389
+ 0.0,
1390
+ 0.0
1391
+ ],
1392
+ "q01": [
1393
+ 0.0,
1394
+ 0.0,
1395
+ 0.0,
1396
+ 0.0,
1397
+ 0.0,
1398
+ 0.0,
1399
+ 0.0
1400
+ ],
1401
+ "q99": [
1402
+ 0.0,
1403
+ 0.0,
1404
+ 0.0,
1405
+ 0.0,
1406
+ 0.0,
1407
+ 0.0,
1408
+ 0.0
1409
+ ],
1410
+ "std": [
1411
+ 0.0,
1412
+ 0.0,
1413
+ 0.0,
1414
+ 0.0,
1415
+ 0.0,
1416
+ 0.0,
1417
+ 0.0
1418
+ ]
1419
+ }
1420
+ },
1421
+ "bridge_orig": {
1422
+ "action": {
1423
+ "mask": [
1424
+ true,
1425
+ true,
1426
+ true,
1427
+ true,
1428
+ true,
1429
+ true,
1430
+ false
1431
+ ],
1432
+ "max": [
1433
+ 0.41691166162490845,
1434
+ 0.25864794850349426,
1435
+ 0.21218234300613403,
1436
+ 3.122201919555664,
1437
+ 1.8618112802505493,
1438
+ 6.280478477478027,
1439
+ 1.0
1440
+ ],
1441
+ "mean": [
1442
+ 0.0002334194869035855,
1443
+ 0.00013004911306779832,
1444
+ -0.00012762474943883717,
1445
+ -0.0001556558854645118,
1446
+ -0.0004039328487124294,
1447
+ 0.00023557482927571982,
1448
+ 0.5764579176902771
1449
+ ],
1450
+ "min": [
1451
+ -0.4007510244846344,
1452
+ -0.13874775171279907,
1453
+ -0.22553899884223938,
1454
+ -3.2010786533355713,
1455
+ -1.8618112802505493,
1456
+ -6.279075622558594,
1457
+ 0.0
1458
+ ],
1459
+ "q01": [
1460
+ -0.02872725307941437,
1461
+ -0.04170349963009357,
1462
+ -0.026093858778476715,
1463
+ -0.08092105075716972,
1464
+ -0.09288699507713317,
1465
+ -0.20718276381492615,
1466
+ 0.0
1467
+ ],
1468
+ "q99": [
1469
+ 0.028309678435325586,
1470
+ 0.040855254605412394,
1471
+ 0.040161586627364146,
1472
+ 0.08192047759890528,
1473
+ 0.07792850524187081,
1474
+ 0.20382574498653397,
1475
+ 1.0
1476
+ ],
1477
+ "std": [
1478
+ 0.009765930473804474,
1479
+ 0.013689135201275349,
1480
+ 0.012667362578213215,
1481
+ 0.028534092009067535,
1482
+ 0.030637972056865692,
1483
+ 0.07691419124603271,
1484
+ 0.4973701536655426
1485
+ ]
1486
+ },
1487
+ "num_trajectories": 60064,
1488
+ "num_transitions": 2135463,
1489
+ "proprio": {
1490
+ "max": [
1491
+ 0.0,
1492
+ 0.0,
1493
+ 0.0,
1494
+ 0.0,
1495
+ 0.0,
1496
+ 0.0,
1497
+ 0.0
1498
+ ],
1499
+ "mean": [
1500
+ 0.0,
1501
+ 0.0,
1502
+ 0.0,
1503
+ 0.0,
1504
+ 0.0,
1505
+ 0.0,
1506
+ 0.0
1507
+ ],
1508
+ "min": [
1509
+ 0.0,
1510
+ 0.0,
1511
+ 0.0,
1512
+ 0.0,
1513
+ 0.0,
1514
+ 0.0,
1515
+ 0.0
1516
+ ],
1517
+ "q01": [
1518
+ 0.0,
1519
+ 0.0,
1520
+ 0.0,
1521
+ 0.0,
1522
+ 0.0,
1523
+ 0.0,
1524
+ 0.0
1525
+ ],
1526
+ "q99": [
1527
+ 0.0,
1528
+ 0.0,
1529
+ 0.0,
1530
+ 0.0,
1531
+ 0.0,
1532
+ 0.0,
1533
+ 0.0
1534
+ ],
1535
+ "std": [
1536
+ 0.0,
1537
+ 0.0,
1538
+ 0.0,
1539
+ 0.0,
1540
+ 0.0,
1541
+ 0.0,
1542
+ 0.0
1543
+ ]
1544
+ }
1545
+ },
1546
+ "cmu_stretch": {
1547
+ "action": {
1548
+ "mask": [
1549
+ true,
1550
+ true,
1551
+ true,
1552
+ true,
1553
+ true,
1554
+ true,
1555
+ false
1556
+ ],
1557
+ "max": [
1558
+ 0.02338407188653946,
1559
+ 0.0,
1560
+ 0.023404927924275398,
1561
+ 0.0,
1562
+ 0.0,
1563
+ 0.0,
1564
+ 1.0
1565
+ ],
1566
+ "mean": [
1567
+ 0.00036304505192674696,
1568
+ 0.0,
1569
+ 0.0016466958913952112,
1570
+ 0.0,
1571
+ 0.0,
1572
+ 0.0,
1573
+ 0.3987048268318176
1574
+ ],
1575
+ "min": [
1576
+ -0.019353797659277916,
1577
+ 0.0,
1578
+ -0.02019215188920498,
1579
+ 0.0,
1580
+ 0.0,
1581
+ 0.0,
1582
+ 0.0
1583
+ ],
1584
+ "q01": [
1585
+ -0.011175686959177256,
1586
+ 0.0,
1587
+ -0.0032206363626755773,
1588
+ 0.0,
1589
+ 0.0,
1590
+ 0.0,
1591
+ 0.0
1592
+ ],
1593
+ "q99": [
1594
+ 0.014501785952597848,
1595
+ 0.0,
1596
+ 0.015056106168776728,
1597
+ 0.0,
1598
+ 0.0,
1599
+ 0.0,
1600
+ 1.0
1601
+ ],
1602
+ "std": [
1603
+ 0.004081828519701958,
1604
+ 0.0,
1605
+ 0.0037743328139185905,
1606
+ 0.0,
1607
+ 0.0,
1608
+ 0.0,
1609
+ 0.48963725566864014
1610
+ ]
1611
+ },
1612
+ "num_trajectories": 135,
1613
+ "num_transitions": 25016,
1614
+ "proprio": {
1615
+ "max": [
1616
+ 0.0,
1617
+ 0.0,
1618
+ 0.0,
1619
+ 0.0,
1620
+ 0.0,
1621
+ 0.0,
1622
+ 0.0
1623
+ ],
1624
+ "mean": [
1625
+ 0.0,
1626
+ 0.0,
1627
+ 0.0,
1628
+ 0.0,
1629
+ 0.0,
1630
+ 0.0,
1631
+ 0.0
1632
+ ],
1633
+ "min": [
1634
+ 0.0,
1635
+ 0.0,
1636
+ 0.0,
1637
+ 0.0,
1638
+ 0.0,
1639
+ 0.0,
1640
+ 0.0
1641
+ ],
1642
+ "q01": [
1643
+ 0.0,
1644
+ 0.0,
1645
+ 0.0,
1646
+ 0.0,
1647
+ 0.0,
1648
+ 0.0,
1649
+ 0.0
1650
+ ],
1651
+ "q99": [
1652
+ 0.0,
1653
+ 0.0,
1654
+ 0.0,
1655
+ 0.0,
1656
+ 0.0,
1657
+ 0.0,
1658
+ 0.0
1659
+ ],
1660
+ "std": [
1661
+ 0.0,
1662
+ 0.0,
1663
+ 0.0,
1664
+ 0.0,
1665
+ 0.0,
1666
+ 0.0,
1667
+ 0.0
1668
+ ]
1669
+ }
1670
+ },
1671
+ "dlr_edan_shared_control_converted_externally_to_rlds": {
1672
+ "action": {
1673
+ "mask": [
1674
+ true,
1675
+ true,
1676
+ true,
1677
+ true,
1678
+ true,
1679
+ true,
1680
+ false
1681
+ ],
1682
+ "max": [
1683
+ 0.18991442024707794,
1684
+ 0.0739002525806427,
1685
+ 0.18064819276332855,
1686
+ 0.0866486132144928,
1687
+ 0.13464981317520142,
1688
+ 0.16910280287265778,
1689
+ 1.0
1690
+ ],
1691
+ "mean": [
1692
+ 0.006647810339927673,
1693
+ -0.0007657372043468058,
1694
+ 0.006522852927446365,
1695
+ 0.0011679717572405934,
1696
+ -0.006395625416189432,
1697
+ -0.011902998201549053,
1698
+ 0.6985887289047241
1699
+ ],
1700
+ "min": [
1701
+ -0.10054297000169754,
1702
+ -0.08427435159683228,
1703
+ -0.13533438742160797,
1704
+ -0.17556548118591309,
1705
+ -0.18485672771930695,
1706
+ -0.2680685818195343,
1707
+ 0.0
1708
+ ],
1709
+ "q01": [
1710
+ -0.02987122368067503,
1711
+ -0.06013262912631035,
1712
+ -0.08286409199237824,
1713
+ -0.05924444157630205,
1714
+ -0.15986866518855095,
1715
+ -0.15636983573436739,
1716
+ 0.0
1717
+ ],
1718
+ "q99": [
1719
+ 0.08832092039287087,
1720
+ 0.042126184627413736,
1721
+ 0.11311905644834042,
1722
+ 0.0643695573508739,
1723
+ 0.03941855944693088,
1724
+ 0.156646853685379,
1725
+ 1.0
1726
+ ],
1727
+ "std": [
1728
+ 0.021393608301877975,
1729
+ 0.01814231649041176,
1730
+ 0.03374375030398369,
1731
+ 0.01743541844189167,
1732
+ 0.03394376486539841,
1733
+ 0.04641875624656677,
1734
+ 0.4588589072227478
1735
+ ]
1736
+ },
1737
+ "num_trajectories": 104,
1738
+ "num_transitions": 8928,
1739
+ "proprio": {
1740
+ "max": [
1741
+ 0.0,
1742
+ 0.0,
1743
+ 0.0,
1744
+ 0.0,
1745
+ 0.0,
1746
+ 0.0,
1747
+ 0.0
1748
+ ],
1749
+ "mean": [
1750
+ 0.0,
1751
+ 0.0,
1752
+ 0.0,
1753
+ 0.0,
1754
+ 0.0,
1755
+ 0.0,
1756
+ 0.0
1757
+ ],
1758
+ "min": [
1759
+ 0.0,
1760
+ 0.0,
1761
+ 0.0,
1762
+ 0.0,
1763
+ 0.0,
1764
+ 0.0,
1765
+ 0.0
1766
+ ],
1767
+ "q01": [
1768
+ 0.0,
1769
+ 0.0,
1770
+ 0.0,
1771
+ 0.0,
1772
+ 0.0,
1773
+ 0.0,
1774
+ 0.0
1775
+ ],
1776
+ "q99": [
1777
+ 0.0,
1778
+ 0.0,
1779
+ 0.0,
1780
+ 0.0,
1781
+ 0.0,
1782
+ 0.0,
1783
+ 0.0
1784
+ ],
1785
+ "std": [
1786
+ 0.0,
1787
+ 0.0,
1788
+ 0.0,
1789
+ 0.0,
1790
+ 0.0,
1791
+ 0.0,
1792
+ 0.0
1793
+ ]
1794
+ }
1795
+ },
1796
+ "dobbe": {
1797
+ "action": {
1798
+ "mask": [
1799
+ true,
1800
+ true,
1801
+ true,
1802
+ true,
1803
+ true,
1804
+ true,
1805
+ false
1806
+ ],
1807
+ "max": [
1808
+ 38.590423583984375,
1809
+ 17.932697296142578,
1810
+ 4.843764305114746,
1811
+ 1.4372116327285767,
1812
+ 0.4340403974056244,
1813
+ 1.2057193517684937,
1814
+ 0.9998947381973267
1815
+ ],
1816
+ "mean": [
1817
+ -0.0001120665911003016,
1818
+ 0.0011229600058868527,
1819
+ -0.00010194431524723768,
1820
+ -7.371398532995954e-05,
1821
+ -0.00067531579406932,
1822
+ -5.6643435527803376e-05,
1823
+ 0.6318281888961792
1824
+ ],
1825
+ "min": [
1826
+ -5.700923442840576,
1827
+ -21.605947494506836,
1828
+ -123.72489929199219,
1829
+ -1.7229845523834229,
1830
+ -0.4998578727245331,
1831
+ -0.8867913484573364,
1832
+ 1.4196479014572105e-06
1833
+ ],
1834
+ "q01": [
1835
+ -0.01119564864784479,
1836
+ -0.014266146533191203,
1837
+ -0.0071747214533388615,
1838
+ -0.009444301575422287,
1839
+ -0.03990109823644161,
1840
+ -0.017422311007976532,
1841
+ 4.003279136668425e-05
1842
+ ],
1843
+ "q99": [
1844
+ 0.01015154086053368,
1845
+ 0.017181577533483497,
1846
+ 0.007216989761218411,
1847
+ 0.010380979906767595,
1848
+ 0.03556173853576176,
1849
+ 0.018032474815845446,
1850
+ 0.9982578039169312
1851
+ ],
1852
+ "std": [
1853
+ 0.04264938458800316,
1854
+ 0.04428559169173241,
1855
+ 0.12224084138870239,
1856
+ 0.005388413090258837,
1857
+ 0.011246449314057827,
1858
+ 0.006287882570177317,
1859
+ 0.39732322096824646
1860
+ ]
1861
+ },
1862
+ "num_trajectories": 5208,
1863
+ "num_transitions": 1139911,
1864
+ "proprio": {
1865
+ "max": [
1866
+ 0.0,
1867
+ 0.0,
1868
+ 0.0,
1869
+ 0.0,
1870
+ 0.0,
1871
+ 0.0,
1872
+ 0.0
1873
+ ],
1874
+ "mean": [
1875
+ 0.0,
1876
+ 0.0,
1877
+ 0.0,
1878
+ 0.0,
1879
+ 0.0,
1880
+ 0.0,
1881
+ 0.0
1882
+ ],
1883
+ "min": [
1884
+ 0.0,
1885
+ 0.0,
1886
+ 0.0,
1887
+ 0.0,
1888
+ 0.0,
1889
+ 0.0,
1890
+ 0.0
1891
+ ],
1892
+ "q01": [
1893
+ 0.0,
1894
+ 0.0,
1895
+ 0.0,
1896
+ 0.0,
1897
+ 0.0,
1898
+ 0.0,
1899
+ 0.0
1900
+ ],
1901
+ "q99": [
1902
+ 0.0,
1903
+ 0.0,
1904
+ 0.0,
1905
+ 0.0,
1906
+ 0.0,
1907
+ 0.0,
1908
+ 0.0
1909
+ ],
1910
+ "std": [
1911
+ 0.0,
1912
+ 0.0,
1913
+ 0.0,
1914
+ 0.0,
1915
+ 0.0,
1916
+ 0.0,
1917
+ 0.0
1918
+ ]
1919
+ }
1920
+ },
1921
+ "fmb_dataset": {
1922
+ "action": {
1923
+ "mask": [
1924
+ true,
1925
+ true,
1926
+ true,
1927
+ true,
1928
+ true,
1929
+ true,
1930
+ false
1931
+ ],
1932
+ "max": [
1933
+ 1.399999976158142,
1934
+ 1.0,
1935
+ 1.399999976158142,
1936
+ 1.0,
1937
+ 1.0,
1938
+ 1.0,
1939
+ 1.0
1940
+ ],
1941
+ "mean": [
1942
+ 0.059029702097177505,
1943
+ -0.06476633995771408,
1944
+ -0.09787475317716599,
1945
+ 0.004325388930737972,
1946
+ 0.00028963794466108084,
1947
+ -0.04457257315516472,
1948
+ 0.7336440086364746
1949
+ ],
1950
+ "min": [
1951
+ -1.399999976158142,
1952
+ -1.399999976158142,
1953
+ -1.0,
1954
+ -1.0,
1955
+ -1.0,
1956
+ -1.0,
1957
+ 0.0
1958
+ ],
1959
+ "q01": [
1960
+ -0.8257142901420593,
1961
+ -1.399999976158142,
1962
+ -1.0,
1963
+ -1.0,
1964
+ -0.3028571307659149,
1965
+ -1.0,
1966
+ 0.0
1967
+ ],
1968
+ "q99": [
1969
+ 1.0,
1970
+ 0.5257142782211304,
1971
+ 1.0,
1972
+ 1.0,
1973
+ 0.3400000035762787,
1974
+ 1.0,
1975
+ 1.0
1976
+ ],
1977
+ "std": [
1978
+ 0.28809213638305664,
1979
+ 0.2820415794849396,
1980
+ 0.4626740515232086,
1981
+ 0.3266514539718628,
1982
+ 0.10842999070882797,
1983
+ 0.3440099358558655,
1984
+ 0.4435282051563263
1985
+ ]
1986
+ },
1987
+ "num_trajectories": 8612,
1988
+ "num_transitions": 1137459,
1989
+ "proprio": {
1990
+ "max": [
1991
+ 0.0,
1992
+ 0.0,
1993
+ 0.0,
1994
+ 0.0,
1995
+ 0.0,
1996
+ 0.0,
1997
+ 0.0
1998
+ ],
1999
+ "mean": [
2000
+ 0.0,
2001
+ 0.0,
2002
+ 0.0,
2003
+ 0.0,
2004
+ 0.0,
2005
+ 0.0,
2006
+ 0.0
2007
+ ],
2008
+ "min": [
2009
+ 0.0,
2010
+ 0.0,
2011
+ 0.0,
2012
+ 0.0,
2013
+ 0.0,
2014
+ 0.0,
2015
+ 0.0
2016
+ ],
2017
+ "q01": [
2018
+ 0.0,
2019
+ 0.0,
2020
+ 0.0,
2021
+ 0.0,
2022
+ 0.0,
2023
+ 0.0,
2024
+ 0.0
2025
+ ],
2026
+ "q99": [
2027
+ 0.0,
2028
+ 0.0,
2029
+ 0.0,
2030
+ 0.0,
2031
+ 0.0,
2032
+ 0.0,
2033
+ 0.0
2034
+ ],
2035
+ "std": [
2036
+ 0.0,
2037
+ 0.0,
2038
+ 0.0,
2039
+ 0.0,
2040
+ 0.0,
2041
+ 0.0,
2042
+ 0.0
2043
+ ]
2044
+ }
2045
+ },
2046
+ "fractal20220817_data": {
2047
+ "action": {
2048
+ "mask": [
2049
+ true,
2050
+ true,
2051
+ true,
2052
+ true,
2053
+ true,
2054
+ true,
2055
+ false
2056
+ ],
2057
+ "max": [
2058
+ 2.9984593391418457,
2059
+ 22.09052848815918,
2060
+ 2.7507524490356445,
2061
+ 1.570636510848999,
2062
+ 1.5321086645126343,
2063
+ 1.5691522359848022,
2064
+ 1.0
2065
+ ],
2066
+ "mean": [
2067
+ 0.006987582892179489,
2068
+ 0.006265917327255011,
2069
+ -0.01262515690177679,
2070
+ 0.04333311319351196,
2071
+ -0.005756212864071131,
2072
+ 0.0009130256366916001,
2073
+ 0.5354204773902893
2074
+ ],
2075
+ "min": [
2076
+ -2.0204520225524902,
2077
+ -5.497899532318115,
2078
+ -2.031663417816162,
2079
+ -1.569917917251587,
2080
+ -1.569892168045044,
2081
+ -1.570419430732727,
2082
+ 0.0
2083
+ ],
2084
+ "q01": [
2085
+ -0.22453527510166169,
2086
+ -0.14820013284683228,
2087
+ -0.231589707583189,
2088
+ -0.3517994859814644,
2089
+ -0.4193011274933815,
2090
+ -0.43643461108207704,
2091
+ 0.0
2092
+ ],
2093
+ "q99": [
2094
+ 0.17824687153100965,
2095
+ 0.14938379630446405,
2096
+ 0.21842354819178575,
2097
+ 0.5892666035890578,
2098
+ 0.35272657424211445,
2099
+ 0.44796681255102094,
2100
+ 1.0
2101
+ ],
2102
+ "std": [
2103
+ 0.0692116990685463,
2104
+ 0.05970962345600128,
2105
+ 0.07353084534406662,
2106
+ 0.15610496699810028,
2107
+ 0.13164450228214264,
2108
+ 0.14593800902366638,
2109
+ 0.497110515832901
2110
+ ]
2111
+ },
2112
+ "num_trajectories": 87212,
2113
+ "num_transitions": 3786400,
2114
+ "proprio": {
2115
+ "max": [
2116
+ 0.0,
2117
+ 0.0,
2118
+ 0.0,
2119
+ 0.0,
2120
+ 0.0,
2121
+ 0.0,
2122
+ 0.0
2123
+ ],
2124
+ "mean": [
2125
+ 0.0,
2126
+ 0.0,
2127
+ 0.0,
2128
+ 0.0,
2129
+ 0.0,
2130
+ 0.0,
2131
+ 0.0
2132
+ ],
2133
+ "min": [
2134
+ 0.0,
2135
+ 0.0,
2136
+ 0.0,
2137
+ 0.0,
2138
+ 0.0,
2139
+ 0.0,
2140
+ 0.0
2141
+ ],
2142
+ "q01": [
2143
+ 0.0,
2144
+ 0.0,
2145
+ 0.0,
2146
+ 0.0,
2147
+ 0.0,
2148
+ 0.0,
2149
+ 0.0
2150
+ ],
2151
+ "q99": [
2152
+ 0.0,
2153
+ 0.0,
2154
+ 0.0,
2155
+ 0.0,
2156
+ 0.0,
2157
+ 0.0,
2158
+ 0.0
2159
+ ],
2160
+ "std": [
2161
+ 0.0,
2162
+ 0.0,
2163
+ 0.0,
2164
+ 0.0,
2165
+ 0.0,
2166
+ 0.0,
2167
+ 0.0
2168
+ ]
2169
+ }
2170
+ },
2171
+ "furniture_bench_dataset_converted_externally_to_rlds": {
2172
+ "action": {
2173
+ "mask": [
2174
+ true,
2175
+ true,
2176
+ true,
2177
+ true,
2178
+ true,
2179
+ true,
2180
+ false
2181
+ ],
2182
+ "max": [
2183
+ 0.10000000149011612,
2184
+ 0.10000000149011612,
2185
+ 0.10000000149011612,
2186
+ 0.8651833534240723,
2187
+ 1.0909736156463623,
2188
+ 2.863185405731201,
2189
+ 1.0
2190
+ ],
2191
+ "mean": [
2192
+ 0.00014610752987209707,
2193
+ 0.0010830952087417245,
2194
+ 0.0006224989192560315,
2195
+ -0.003303206292912364,
2196
+ -0.0026880695950239897,
2197
+ 0.018242603167891502,
2198
+ 0.48854944109916687
2199
+ ],
2200
+ "min": [
2201
+ -0.10495579987764359,
2202
+ -0.10939455777406693,
2203
+ -0.10000000149011612,
2204
+ -0.971906840801239,
2205
+ -1.0475432872772217,
2206
+ -3.06000018119812,
2207
+ 0.0
2208
+ ],
2209
+ "q01": [
2210
+ -0.053988199681043625,
2211
+ -0.05049169331789017,
2212
+ -0.032499241530895236,
2213
+ -0.1953887003660202,
2214
+ -0.41674559473991396,
2215
+ -0.8886768388748169,
2216
+ 0.0
2217
+ ],
2218
+ "q99": [
2219
+ 0.05414841488003723,
2220
+ 0.04965164884924884,
2221
+ 0.060055799782276154,
2222
+ 0.18231668293476103,
2223
+ 0.39867786407470646,
2224
+ 0.8772023963928218,
2225
+ 1.0
2226
+ ],
2227
+ "std": [
2228
+ 0.01610708422958851,
2229
+ 0.014891477301716805,
2230
+ 0.014014219865202904,
2231
+ 0.058274295181035995,
2232
+ 0.11417088657617569,
2233
+ 0.33479776978492737,
2234
+ 0.49991825222969055
2235
+ ]
2236
+ },
2237
+ "num_trajectories": 5100,
2238
+ "num_transitions": 3948057,
2239
+ "proprio": {
2240
+ "max": [
2241
+ 0.0,
2242
+ 0.0,
2243
+ 0.0,
2244
+ 0.0,
2245
+ 0.0,
2246
+ 0.0,
2247
+ 0.0
2248
+ ],
2249
+ "mean": [
2250
+ 0.0,
2251
+ 0.0,
2252
+ 0.0,
2253
+ 0.0,
2254
+ 0.0,
2255
+ 0.0,
2256
+ 0.0
2257
+ ],
2258
+ "min": [
2259
+ 0.0,
2260
+ 0.0,
2261
+ 0.0,
2262
+ 0.0,
2263
+ 0.0,
2264
+ 0.0,
2265
+ 0.0
2266
+ ],
2267
+ "q01": [
2268
+ 0.0,
2269
+ 0.0,
2270
+ 0.0,
2271
+ 0.0,
2272
+ 0.0,
2273
+ 0.0,
2274
+ 0.0
2275
+ ],
2276
+ "q99": [
2277
+ 0.0,
2278
+ 0.0,
2279
+ 0.0,
2280
+ 0.0,
2281
+ 0.0,
2282
+ 0.0,
2283
+ 0.0
2284
+ ],
2285
+ "std": [
2286
+ 0.0,
2287
+ 0.0,
2288
+ 0.0,
2289
+ 0.0,
2290
+ 0.0,
2291
+ 0.0,
2292
+ 0.0
2293
+ ]
2294
+ }
2295
+ },
2296
+ "iamlab_cmu_pickup_insert_converted_externally_to_rlds": {
2297
+ "action": {
2298
+ "mask": [
2299
+ true,
2300
+ true,
2301
+ true,
2302
+ true,
2303
+ true,
2304
+ true,
2305
+ false
2306
+ ],
2307
+ "max": [
2308
+ 0.6634981632232666,
2309
+ 0.23428471386432648,
2310
+ 0.4308285415172577,
2311
+ 3.1415927410125732,
2312
+ 0.13647015392780304,
2313
+ 3.141592502593994,
2314
+ 1.0
2315
+ ],
2316
+ "mean": [
2317
+ 0.5274372696876526,
2318
+ 0.02858201041817665,
2319
+ 0.18712575733661652,
2320
+ 1.2339589595794678,
2321
+ 0.03226623684167862,
2322
+ -1.4199490547180176,
2323
+ 0.5550631880760193
2324
+ ],
2325
+ "min": [
2326
+ 0.3071657121181488,
2327
+ -0.29754969477653503,
2328
+ 0.06578229367733002,
2329
+ -3.1415927410125732,
2330
+ -0.04584203287959099,
2331
+ -3.141592502593994,
2332
+ 0.0
2333
+ ],
2334
+ "q01": [
2335
+ 0.3148897051811218,
2336
+ -0.20317550599575043,
2337
+ 0.06785467118024827,
2338
+ -3.140952730178833,
2339
+ -0.029743434861302376,
2340
+ -3.141091251373291,
2341
+ 0.0
2342
+ ],
2343
+ "q99": [
2344
+ 0.6472805738449097,
2345
+ 0.20846802592277527,
2346
+ 0.36855655312538155,
2347
+ 3.1409926891326903,
2348
+ 0.11424950212240226,
2349
+ 3.1410969257354737,
2350
+ 1.0
2351
+ ],
2352
+ "std": [
2353
+ 0.08108345419168472,
2354
+ 0.1116757020354271,
2355
+ 0.07747554779052734,
2356
+ 2.8737246990203857,
2357
+ 0.02774704433977604,
2358
+ 2.7678682804107666,
2359
+ 0.49695101380348206
2360
+ ]
2361
+ },
2362
+ "num_trajectories": 631,
2363
+ "num_transitions": 146241,
2364
+ "proprio": {
2365
+ "max": [
2366
+ 0.0,
2367
+ 0.0,
2368
+ 0.0,
2369
+ 0.0,
2370
+ 0.0,
2371
+ 0.0,
2372
+ 0.0
2373
+ ],
2374
+ "mean": [
2375
+ 0.0,
2376
+ 0.0,
2377
+ 0.0,
2378
+ 0.0,
2379
+ 0.0,
2380
+ 0.0,
2381
+ 0.0
2382
+ ],
2383
+ "min": [
2384
+ 0.0,
2385
+ 0.0,
2386
+ 0.0,
2387
+ 0.0,
2388
+ 0.0,
2389
+ 0.0,
2390
+ 0.0
2391
+ ],
2392
+ "q01": [
2393
+ 0.0,
2394
+ 0.0,
2395
+ 0.0,
2396
+ 0.0,
2397
+ 0.0,
2398
+ 0.0,
2399
+ 0.0
2400
+ ],
2401
+ "q99": [
2402
+ 0.0,
2403
+ 0.0,
2404
+ 0.0,
2405
+ 0.0,
2406
+ 0.0,
2407
+ 0.0,
2408
+ 0.0
2409
+ ],
2410
+ "std": [
2411
+ 0.0,
2412
+ 0.0,
2413
+ 0.0,
2414
+ 0.0,
2415
+ 0.0,
2416
+ 0.0,
2417
+ 0.0
2418
+ ]
2419
+ }
2420
+ },
2421
+ "jaco_play": {
2422
+ "action": {
2423
+ "mask": [
2424
+ true,
2425
+ true,
2426
+ true,
2427
+ true,
2428
+ true,
2429
+ true,
2430
+ false
2431
+ ],
2432
+ "max": [
2433
+ 0.20000000298023224,
2434
+ 0.20000000298023224,
2435
+ 0.20000000298023224,
2436
+ 0.0,
2437
+ 0.0,
2438
+ 0.0,
2439
+ 1.0
2440
+ ],
2441
+ "mean": [
2442
+ 0.0009658430935814977,
2443
+ -0.00580078037455678,
2444
+ -0.00395062193274498,
2445
+ 0.0,
2446
+ 0.0,
2447
+ 0.0,
2448
+ 0.34934908151626587
2449
+ ],
2450
+ "min": [
2451
+ -0.20000000298023224,
2452
+ -0.20000000298023224,
2453
+ -0.20000000298023224,
2454
+ 0.0,
2455
+ 0.0,
2456
+ 0.0,
2457
+ 0.0
2458
+ ],
2459
+ "q01": [
2460
+ -0.20000000298023224,
2461
+ -0.20000000298023224,
2462
+ -0.20000000298023224,
2463
+ 0.0,
2464
+ 0.0,
2465
+ 0.0,
2466
+ 0.0
2467
+ ],
2468
+ "q99": [
2469
+ 0.20000000298023224,
2470
+ 0.20000000298023224,
2471
+ 0.20000000298023224,
2472
+ 0.0,
2473
+ 0.0,
2474
+ 0.0,
2475
+ 1.0
2476
+ ],
2477
+ "std": [
2478
+ 0.12235074490308762,
2479
+ 0.09678777307271957,
2480
+ 0.11155334860086441,
2481
+ 0.0,
2482
+ 0.0,
2483
+ 0.0,
2484
+ 0.4768252968788147
2485
+ ]
2486
+ },
2487
+ "num_trajectories": 1085,
2488
+ "num_transitions": 77965,
2489
+ "proprio": {
2490
+ "max": [
2491
+ 0.0,
2492
+ 0.0,
2493
+ 0.0,
2494
+ 0.0,
2495
+ 0.0,
2496
+ 0.0,
2497
+ 0.0
2498
+ ],
2499
+ "mean": [
2500
+ 0.0,
2501
+ 0.0,
2502
+ 0.0,
2503
+ 0.0,
2504
+ 0.0,
2505
+ 0.0,
2506
+ 0.0
2507
+ ],
2508
+ "min": [
2509
+ 0.0,
2510
+ 0.0,
2511
+ 0.0,
2512
+ 0.0,
2513
+ 0.0,
2514
+ 0.0,
2515
+ 0.0
2516
+ ],
2517
+ "q01": [
2518
+ 0.0,
2519
+ 0.0,
2520
+ 0.0,
2521
+ 0.0,
2522
+ 0.0,
2523
+ 0.0,
2524
+ 0.0
2525
+ ],
2526
+ "q99": [
2527
+ 0.0,
2528
+ 0.0,
2529
+ 0.0,
2530
+ 0.0,
2531
+ 0.0,
2532
+ 0.0,
2533
+ 0.0
2534
+ ],
2535
+ "std": [
2536
+ 0.0,
2537
+ 0.0,
2538
+ 0.0,
2539
+ 0.0,
2540
+ 0.0,
2541
+ 0.0,
2542
+ 0.0
2543
+ ]
2544
+ }
2545
+ },
2546
+ "kuka": {
2547
+ "action": {
2548
+ "mask": [
2549
+ true,
2550
+ true,
2551
+ true,
2552
+ true,
2553
+ true,
2554
+ true,
2555
+ false
2556
+ ],
2557
+ "max": [
2558
+ 0.1697135865688324,
2559
+ 0.2777623236179352,
2560
+ 0.43710532784461975,
2561
+ 0.0,
2562
+ 0.0,
2563
+ 1.9684287309646606,
2564
+ 1.0
2565
+ ],
2566
+ "mean": [
2567
+ -0.0004668905457947403,
2568
+ 0.00040138536132872105,
2569
+ -0.001280792523175478,
2570
+ 0.0,
2571
+ 0.0,
2572
+ -0.03722453489899635,
2573
+ 0.4131543040275574
2574
+ ],
2575
+ "min": [
2576
+ -0.159867063164711,
2577
+ -0.2892282009124756,
2578
+ -0.2795473635196686,
2579
+ 0.0,
2580
+ 0.0,
2581
+ -1.9875637292861938,
2582
+ 0.0
2583
+ ],
2584
+ "q01": [
2585
+ -0.06619441494345665,
2586
+ -0.08713878810405731,
2587
+ -0.15083016991615295,
2588
+ 0.0,
2589
+ 0.0,
2590
+ -0.5415697038173676,
2591
+ 0.0
2592
+ ],
2593
+ "q99": [
2594
+ 0.06601839080452929,
2595
+ 0.08732476785779003,
2596
+ 0.18168179214000715,
2597
+ 0.0,
2598
+ 0.0,
2599
+ 0.2923380345106127,
2600
+ 1.0
2601
+ ],
2602
+ "std": [
2603
+ 0.02083250693976879,
2604
+ 0.02915887162089348,
2605
+ 0.06422865390777588,
2606
+ 0.0,
2607
+ 0.0,
2608
+ 0.14224295318126678,
2609
+ 0.49086448550224304
2610
+ ]
2611
+ },
2612
+ "num_trajectories": 209880,
2613
+ "num_transitions": 2455879,
2614
+ "proprio": {
2615
+ "max": [
2616
+ 0.0,
2617
+ 0.0,
2618
+ 0.0,
2619
+ 0.0,
2620
+ 0.0,
2621
+ 0.0,
2622
+ 0.0
2623
+ ],
2624
+ "mean": [
2625
+ 0.0,
2626
+ 0.0,
2627
+ 0.0,
2628
+ 0.0,
2629
+ 0.0,
2630
+ 0.0,
2631
+ 0.0
2632
+ ],
2633
+ "min": [
2634
+ 0.0,
2635
+ 0.0,
2636
+ 0.0,
2637
+ 0.0,
2638
+ 0.0,
2639
+ 0.0,
2640
+ 0.0
2641
+ ],
2642
+ "q01": [
2643
+ 0.0,
2644
+ 0.0,
2645
+ 0.0,
2646
+ 0.0,
2647
+ 0.0,
2648
+ 0.0,
2649
+ 0.0
2650
+ ],
2651
+ "q99": [
2652
+ 0.0,
2653
+ 0.0,
2654
+ 0.0,
2655
+ 0.0,
2656
+ 0.0,
2657
+ 0.0,
2658
+ 0.0
2659
+ ],
2660
+ "std": [
2661
+ 0.0,
2662
+ 0.0,
2663
+ 0.0,
2664
+ 0.0,
2665
+ 0.0,
2666
+ 0.0,
2667
+ 0.0
2668
+ ]
2669
+ }
2670
+ },
2671
+ "nyu_franka_play_dataset_converted_externally_to_rlds": {
2672
+ "action": {
2673
+ "mask": [
2674
+ true,
2675
+ true,
2676
+ true,
2677
+ true,
2678
+ true,
2679
+ true,
2680
+ false
2681
+ ],
2682
+ "max": [
2683
+ 0.06424188613891602,
2684
+ 0.07027634978294373,
2685
+ 0.06129661202430725,
2686
+ 6.281067848205566,
2687
+ 0.1967729926109314,
2688
+ 0.26377415657043457,
2689
+ 1.0
2690
+ ],
2691
+ "mean": [
2692
+ 0.001021989737637341,
2693
+ -0.00012002651783404872,
2694
+ 0.00032894269679673016,
2695
+ 0.0015034361276775599,
2696
+ -0.002198522910475731,
2697
+ -0.001663230243138969,
2698
+ 0.7230083346366882
2699
+ ],
2700
+ "min": [
2701
+ -0.05952230095863342,
2702
+ -0.07232445478439331,
2703
+ -0.06730806827545166,
2704
+ -6.278434753417969,
2705
+ -0.21479034423828125,
2706
+ -0.3627619743347168,
2707
+ 0.0
2708
+ ],
2709
+ "q01": [
2710
+ -0.03199600875377655,
2711
+ -0.032861671447753905,
2712
+ -0.03368805110454559,
2713
+ -0.12080862045288086,
2714
+ -0.12175218224525451,
2715
+ -0.11370223641395569,
2716
+ 0.0
2717
+ ],
2718
+ "q99": [
2719
+ 0.03101520001888276,
2720
+ 0.0373908892273903,
2721
+ 0.03646374464035038,
2722
+ 0.11764093399047852,
2723
+ 0.1258920183777809,
2724
+ 0.09366151213645942,
2725
+ 1.0
2726
+ ],
2727
+ "std": [
2728
+ 0.01327415369451046,
2729
+ 0.013215910643339157,
2730
+ 0.012822109274566174,
2731
+ 0.2732451558113098,
2732
+ 0.057022541761398315,
2733
+ 0.039172880351543427,
2734
+ 0.44752755761146545
2735
+ ]
2736
+ },
2737
+ "num_trajectories": 456,
2738
+ "num_transitions": 44875,
2739
+ "proprio": {
2740
+ "max": [
2741
+ 0.0,
2742
+ 0.0,
2743
+ 0.0,
2744
+ 0.0,
2745
+ 0.0,
2746
+ 0.0,
2747
+ 0.0
2748
+ ],
2749
+ "mean": [
2750
+ 0.0,
2751
+ 0.0,
2752
+ 0.0,
2753
+ 0.0,
2754
+ 0.0,
2755
+ 0.0,
2756
+ 0.0
2757
+ ],
2758
+ "min": [
2759
+ 0.0,
2760
+ 0.0,
2761
+ 0.0,
2762
+ 0.0,
2763
+ 0.0,
2764
+ 0.0,
2765
+ 0.0
2766
+ ],
2767
+ "q01": [
2768
+ 0.0,
2769
+ 0.0,
2770
+ 0.0,
2771
+ 0.0,
2772
+ 0.0,
2773
+ 0.0,
2774
+ 0.0
2775
+ ],
2776
+ "q99": [
2777
+ 0.0,
2778
+ 0.0,
2779
+ 0.0,
2780
+ 0.0,
2781
+ 0.0,
2782
+ 0.0,
2783
+ 0.0
2784
+ ],
2785
+ "std": [
2786
+ 0.0,
2787
+ 0.0,
2788
+ 0.0,
2789
+ 0.0,
2790
+ 0.0,
2791
+ 0.0,
2792
+ 0.0
2793
+ ]
2794
+ }
2795
+ },
2796
+ "roboturk": {
2797
+ "action": {
2798
+ "mask": [
2799
+ true,
2800
+ true,
2801
+ true,
2802
+ true,
2803
+ true,
2804
+ true,
2805
+ false
2806
+ ],
2807
+ "max": [
2808
+ 0.39124172925949097,
2809
+ 0.4601028263568878,
2810
+ 0.4870833456516266,
2811
+ 1.816888689994812,
2812
+ 1.8240282535552979,
2813
+ 1.4824820756912231,
2814
+ 1.0
2815
+ ],
2816
+ "mean": [
2817
+ 0.0014448732836171985,
2818
+ -0.0015945249469950795,
2819
+ -0.0011753785656765103,
2820
+ 0.0023012510500848293,
2821
+ -0.0009382463176734746,
2822
+ -0.00011485807772260159,
2823
+ 0.5746025443077087
2824
+ ],
2825
+ "min": [
2826
+ -0.6546999216079712,
2827
+ -0.6365841031074524,
2828
+ -0.4217723608016968,
2829
+ -1.6695482730865479,
2830
+ -1.8023357391357422,
2831
+ -1.4630827903747559,
2832
+ 0.0
2833
+ ],
2834
+ "q01": [
2835
+ -0.1342635464668274,
2836
+ -0.19996687173843383,
2837
+ -0.1482972100377083,
2838
+ -0.20720748245716095,
2839
+ -0.09676413893699647,
2840
+ -0.18075634717941286,
2841
+ 0.0
2842
+ ],
2843
+ "q99": [
2844
+ 0.14956976801157001,
2845
+ 0.1805950567126275,
2846
+ 0.18841815620660796,
2847
+ 0.21615413755178453,
2848
+ 0.09457383215427405,
2849
+ 0.18543301910162005,
2850
+ 1.0
2851
+ ],
2852
+ "std": [
2853
+ 0.04935386776924133,
2854
+ 0.0635455846786499,
2855
+ 0.061164740473032,
2856
+ 0.09553450345993042,
2857
+ 0.08420111238956451,
2858
+ 0.06517903506755829,
2859
+ 0.49452081322669983
2860
+ ]
2861
+ },
2862
+ "num_trajectories": 1995,
2863
+ "num_transitions": 187507,
2864
+ "proprio": {
2865
+ "max": [
2866
+ 0.0,
2867
+ 0.0,
2868
+ 0.0,
2869
+ 0.0,
2870
+ 0.0,
2871
+ 0.0,
2872
+ 0.0
2873
+ ],
2874
+ "mean": [
2875
+ 0.0,
2876
+ 0.0,
2877
+ 0.0,
2878
+ 0.0,
2879
+ 0.0,
2880
+ 0.0,
2881
+ 0.0
2882
+ ],
2883
+ "min": [
2884
+ 0.0,
2885
+ 0.0,
2886
+ 0.0,
2887
+ 0.0,
2888
+ 0.0,
2889
+ 0.0,
2890
+ 0.0
2891
+ ],
2892
+ "q01": [
2893
+ 0.0,
2894
+ 0.0,
2895
+ 0.0,
2896
+ 0.0,
2897
+ 0.0,
2898
+ 0.0,
2899
+ 0.0
2900
+ ],
2901
+ "q99": [
2902
+ 0.0,
2903
+ 0.0,
2904
+ 0.0,
2905
+ 0.0,
2906
+ 0.0,
2907
+ 0.0,
2908
+ 0.0
2909
+ ],
2910
+ "std": [
2911
+ 0.0,
2912
+ 0.0,
2913
+ 0.0,
2914
+ 0.0,
2915
+ 0.0,
2916
+ 0.0,
2917
+ 0.0
2918
+ ]
2919
+ }
2920
+ },
2921
+ "stanford_hydra_dataset_converted_externally_to_rlds": {
2922
+ "action": {
2923
+ "mask": [
2924
+ true,
2925
+ true,
2926
+ true,
2927
+ true,
2928
+ true,
2929
+ true,
2930
+ false
2931
+ ],
2932
+ "max": [
2933
+ 0.02499854564666748,
2934
+ 0.02499903365969658,
2935
+ 0.024999922141432762,
2936
+ 0.24974457919597626,
2937
+ 0.24997030198574066,
2938
+ 0.24999946355819702,
2939
+ 1.0
2940
+ ],
2941
+ "mean": [
2942
+ 0.0007790001109242439,
2943
+ 0.00013707754260394722,
2944
+ -0.0002548607881180942,
2945
+ 0.0012903271708637476,
2946
+ -0.004751681815832853,
2947
+ 0.002692886395379901,
2948
+ 0.48855218291282654
2949
+ ],
2950
+ "min": [
2951
+ -0.024999044835567474,
2952
+ -0.024999700486660004,
2953
+ -0.02499929815530777,
2954
+ -0.24993225932121277,
2955
+ -0.2499666064977646,
2956
+ -0.2499932497739792,
2957
+ 0.0
2958
+ ],
2959
+ "q01": [
2960
+ -0.019992006458342076,
2961
+ -0.02415412735193968,
2962
+ -0.022941758055239916,
2963
+ -0.11085530579090118,
2964
+ -0.12024572037160397,
2965
+ -0.13314770206809043,
2966
+ 0.0
2967
+ ],
2968
+ "q99": [
2969
+ 0.022886231057345868,
2970
+ 0.022358838934451335,
2971
+ 0.02410089675337076,
2972
+ 0.12370114490389822,
2973
+ 0.11323311634361738,
2974
+ 0.18474749639630164,
2975
+ 1.0
2976
+ ],
2977
+ "std": [
2978
+ 0.008022161200642586,
2979
+ 0.009131459519267082,
2980
+ 0.009574338793754578,
2981
+ 0.04122216999530792,
2982
+ 0.0384303517639637,
2983
+ 0.04606688767671585,
2984
+ 0.49976691603660583
2985
+ ]
2986
+ },
2987
+ "num_trajectories": 570,
2988
+ "num_transitions": 358234,
2989
+ "proprio": {
2990
+ "max": [
2991
+ 0.0,
2992
+ 0.0,
2993
+ 0.0,
2994
+ 0.0,
2995
+ 0.0,
2996
+ 0.0,
2997
+ 0.0
2998
+ ],
2999
+ "mean": [
3000
+ 0.0,
3001
+ 0.0,
3002
+ 0.0,
3003
+ 0.0,
3004
+ 0.0,
3005
+ 0.0,
3006
+ 0.0
3007
+ ],
3008
+ "min": [
3009
+ 0.0,
3010
+ 0.0,
3011
+ 0.0,
3012
+ 0.0,
3013
+ 0.0,
3014
+ 0.0,
3015
+ 0.0
3016
+ ],
3017
+ "q01": [
3018
+ 0.0,
3019
+ 0.0,
3020
+ 0.0,
3021
+ 0.0,
3022
+ 0.0,
3023
+ 0.0,
3024
+ 0.0
3025
+ ],
3026
+ "q99": [
3027
+ 0.0,
3028
+ 0.0,
3029
+ 0.0,
3030
+ 0.0,
3031
+ 0.0,
3032
+ 0.0,
3033
+ 0.0
3034
+ ],
3035
+ "std": [
3036
+ 0.0,
3037
+ 0.0,
3038
+ 0.0,
3039
+ 0.0,
3040
+ 0.0,
3041
+ 0.0,
3042
+ 0.0
3043
+ ]
3044
+ }
3045
+ },
3046
+ "taco_play": {
3047
+ "action": {
3048
+ "mask": [
3049
+ true,
3050
+ true,
3051
+ true,
3052
+ true,
3053
+ true,
3054
+ true,
3055
+ false
3056
+ ],
3057
+ "max": [
3058
+ 1.4915844202041626,
3059
+ 2.1842432022094727,
3060
+ 2.6836395263671875,
3061
+ 5.035226821899414,
3062
+ 2.665864944458008,
3063
+ 4.250768661499023,
3064
+ 1.0
3065
+ ],
3066
+ "mean": [
3067
+ -0.003845922416076064,
3068
+ 0.009671456180512905,
3069
+ 0.012780580669641495,
3070
+ -0.005403771996498108,
3071
+ -0.009606587700545788,
3072
+ -0.002480733208358288,
3073
+ 0.4263913035392761
3074
+ ],
3075
+ "min": [
3076
+ -4.242457866668701,
3077
+ -3.192805051803589,
3078
+ -1.3371467590332031,
3079
+ -4.202683448791504,
3080
+ -2.6722638607025146,
3081
+ -3.3467135429382324,
3082
+ 0.0
3083
+ ],
3084
+ "q01": [
3085
+ -0.7106140398979186,
3086
+ -1.056944659948349,
3087
+ -0.5878450274467468,
3088
+ -0.7682853937149048,
3089
+ -0.7180147767066956,
3090
+ -1.5527938604354858,
3091
+ 0.0
3092
+ ],
3093
+ "q99": [
3094
+ 0.6482916426658629,
3095
+ 1.0051310062408447,
3096
+ 0.9480248689651489,
3097
+ 0.6926478147506714,
3098
+ 0.6351067513227462,
3099
+ 1.628010264635086,
3100
+ 1.0
3101
+ ],
3102
+ "std": [
3103
+ 0.23254038393497467,
3104
+ 0.36298269033432007,
3105
+ 0.28692901134490967,
3106
+ 0.2617705166339874,
3107
+ 0.2438892275094986,
3108
+ 0.5216503143310547,
3109
+ 0.4946896731853485
3110
+ ]
3111
+ },
3112
+ "num_trajectories": 3603,
3113
+ "num_transitions": 237798,
3114
+ "proprio": {
3115
+ "max": [
3116
+ 0.0,
3117
+ 0.0,
3118
+ 0.0,
3119
+ 0.0,
3120
+ 0.0,
3121
+ 0.0,
3122
+ 0.0
3123
+ ],
3124
+ "mean": [
3125
+ 0.0,
3126
+ 0.0,
3127
+ 0.0,
3128
+ 0.0,
3129
+ 0.0,
3130
+ 0.0,
3131
+ 0.0
3132
+ ],
3133
+ "min": [
3134
+ 0.0,
3135
+ 0.0,
3136
+ 0.0,
3137
+ 0.0,
3138
+ 0.0,
3139
+ 0.0,
3140
+ 0.0
3141
+ ],
3142
+ "q01": [
3143
+ 0.0,
3144
+ 0.0,
3145
+ 0.0,
3146
+ 0.0,
3147
+ 0.0,
3148
+ 0.0,
3149
+ 0.0
3150
+ ],
3151
+ "q99": [
3152
+ 0.0,
3153
+ 0.0,
3154
+ 0.0,
3155
+ 0.0,
3156
+ 0.0,
3157
+ 0.0,
3158
+ 0.0
3159
+ ],
3160
+ "std": [
3161
+ 0.0,
3162
+ 0.0,
3163
+ 0.0,
3164
+ 0.0,
3165
+ 0.0,
3166
+ 0.0,
3167
+ 0.0
3168
+ ]
3169
+ }
3170
+ },
3171
+ "toto": {
3172
+ "action": {
3173
+ "mask": [
3174
+ true,
3175
+ true,
3176
+ true,
3177
+ true,
3178
+ true,
3179
+ true,
3180
+ false
3181
+ ],
3182
+ "max": [
3183
+ 0.6839867234230042,
3184
+ 0.4454185664653778,
3185
+ 0.7984078526496887,
3186
+ 2.120781660079956,
3187
+ 1.371164321899414,
3188
+ 1.4118704795837402,
3189
+ 0.0
3190
+ ],
3191
+ "mean": [
3192
+ 0.38542115688323975,
3193
+ 0.007769413758069277,
3194
+ 0.3632740378379822,
3195
+ -0.6652036905288696,
3196
+ 0.1890396922826767,
3197
+ 0.03298724442720413,
3198
+ 0.0
3199
+ ],
3200
+ "min": [
3201
+ 0.09922284632921219,
3202
+ -0.5180193781852722,
3203
+ 0.13791072368621826,
3204
+ -2.635117530822754,
3205
+ -1.0734480619430542,
3206
+ -1.9282547235488892,
3207
+ 0.0
3208
+ ],
3209
+ "q01": [
3210
+ 0.1756722891330719,
3211
+ -0.3077590811252594,
3212
+ 0.235383919775486,
3213
+ -2.0908505964279174,
3214
+ -0.6191593289375306,
3215
+ -0.7488683319091797,
3216
+ 0.0
3217
+ ],
3218
+ "q99": [
3219
+ 0.6136963081359863,
3220
+ 0.33704194784164443,
3221
+ 0.6681221985816956,
3222
+ 0.7422861719131538,
3223
+ 0.7955395007133507,
3224
+ 0.740464625358582,
3225
+ 0.0
3226
+ ],
3227
+ "std": [
3228
+ 0.12211652100086212,
3229
+ 0.19378550350666046,
3230
+ 0.10178236663341522,
3231
+ 0.5725259184837341,
3232
+ 0.29884573817253113,
3233
+ 0.3259911835193634,
3234
+ 0.0
3235
+ ]
3236
+ },
3237
+ "num_trajectories": 1003,
3238
+ "num_transitions": 325699,
3239
+ "proprio": {
3240
+ "max": [
3241
+ 0.0,
3242
+ 0.0,
3243
+ 0.0,
3244
+ 0.0,
3245
+ 0.0,
3246
+ 0.0,
3247
+ 0.0
3248
+ ],
3249
+ "mean": [
3250
+ 0.0,
3251
+ 0.0,
3252
+ 0.0,
3253
+ 0.0,
3254
+ 0.0,
3255
+ 0.0,
3256
+ 0.0
3257
+ ],
3258
+ "min": [
3259
+ 0.0,
3260
+ 0.0,
3261
+ 0.0,
3262
+ 0.0,
3263
+ 0.0,
3264
+ 0.0,
3265
+ 0.0
3266
+ ],
3267
+ "q01": [
3268
+ 0.0,
3269
+ 0.0,
3270
+ 0.0,
3271
+ 0.0,
3272
+ 0.0,
3273
+ 0.0,
3274
+ 0.0
3275
+ ],
3276
+ "q99": [
3277
+ 0.0,
3278
+ 0.0,
3279
+ 0.0,
3280
+ 0.0,
3281
+ 0.0,
3282
+ 0.0,
3283
+ 0.0
3284
+ ],
3285
+ "std": [
3286
+ 0.0,
3287
+ 0.0,
3288
+ 0.0,
3289
+ 0.0,
3290
+ 0.0,
3291
+ 0.0,
3292
+ 0.0
3293
+ ]
3294
+ }
3295
+ },
3296
+ "ucsd_kitchen_dataset_converted_externally_to_rlds": {
3297
+ "action": {
3298
+ "mask": [
3299
+ true,
3300
+ true,
3301
+ true,
3302
+ true,
3303
+ true,
3304
+ true,
3305
+ false
3306
+ ],
3307
+ "max": [
3308
+ 678.0,
3309
+ 400.0,
3310
+ 507.0,
3311
+ 180.00001525878906,
3312
+ 6.000013828277588,
3313
+ 116.99998474121094,
3314
+ 1.0
3315
+ ],
3316
+ "mean": [
3317
+ 410.37567138671875,
3318
+ 116.9518814086914,
3319
+ 192.35032653808594,
3320
+ -121.22441864013672,
3321
+ -33.84893035888672,
3322
+ 50.016136169433594,
3323
+ 0.741813600063324
3324
+ ],
3325
+ "min": [
3326
+ 172.0,
3327
+ -166.0,
3328
+ -99.99999237060547,
3329
+ -180.00001525878906,
3330
+ -89.0,
3331
+ -96.00010681152344,
3332
+ 0.0
3333
+ ],
3334
+ "q01": [
3335
+ 200.00001052856445,
3336
+ -102.31004211425781,
3337
+ -94.99993370056153,
3338
+ -180.00001525878906,
3339
+ -88.00001525878906,
3340
+ -38.999977111816406,
3341
+ 0.0
3342
+ ],
3343
+ "q99": [
3344
+ 637.0,
3345
+ 368.30999999999995,
3346
+ 493.0,
3347
+ 180.00001525878906,
3348
+ 0.999983012676239,
3349
+ 105.00001525878906,
3350
+ 1.0
3351
+ ],
3352
+ "std": [
3353
+ 122.81494903564453,
3354
+ 108.8009033203125,
3355
+ 130.303466796875,
3356
+ 116.28205108642578,
3357
+ 27.621843338012695,
3358
+ 41.02094650268555,
3359
+ 0.43763357400894165
3360
+ ]
3361
+ },
3362
+ "num_trajectories": 150,
3363
+ "num_transitions": 3970,
3364
+ "proprio": {
3365
+ "max": [
3366
+ 0.0,
3367
+ 0.0,
3368
+ 0.0,
3369
+ 0.0,
3370
+ 0.0,
3371
+ 0.0,
3372
+ 0.0
3373
+ ],
3374
+ "mean": [
3375
+ 0.0,
3376
+ 0.0,
3377
+ 0.0,
3378
+ 0.0,
3379
+ 0.0,
3380
+ 0.0,
3381
+ 0.0
3382
+ ],
3383
+ "min": [
3384
+ 0.0,
3385
+ 0.0,
3386
+ 0.0,
3387
+ 0.0,
3388
+ 0.0,
3389
+ 0.0,
3390
+ 0.0
3391
+ ],
3392
+ "q01": [
3393
+ 0.0,
3394
+ 0.0,
3395
+ 0.0,
3396
+ 0.0,
3397
+ 0.0,
3398
+ 0.0,
3399
+ 0.0
3400
+ ],
3401
+ "q99": [
3402
+ 0.0,
3403
+ 0.0,
3404
+ 0.0,
3405
+ 0.0,
3406
+ 0.0,
3407
+ 0.0,
3408
+ 0.0
3409
+ ],
3410
+ "std": [
3411
+ 0.0,
3412
+ 0.0,
3413
+ 0.0,
3414
+ 0.0,
3415
+ 0.0,
3416
+ 0.0,
3417
+ 0.0
3418
+ ]
3419
+ }
3420
+ },
3421
+ "utaustin_mutex": {
3422
+ "action": {
3423
+ "mask": [
3424
+ true,
3425
+ true,
3426
+ true,
3427
+ true,
3428
+ true,
3429
+ true,
3430
+ false
3431
+ ],
3432
+ "max": [
3433
+ 1.0,
3434
+ 1.0,
3435
+ 1.0,
3436
+ 0.375,
3437
+ 0.375,
3438
+ 0.375,
3439
+ 1.0
3440
+ ],
3441
+ "mean": [
3442
+ 0.06176406890153885,
3443
+ -0.005005486309528351,
3444
+ 0.10216785222291946,
3445
+ -0.03314131125807762,
3446
+ 0.013895004987716675,
3447
+ -0.011317633092403412,
3448
+ 0.5038976669311523
3449
+ ],
3450
+ "min": [
3451
+ -1.0,
3452
+ -1.0,
3453
+ -1.0,
3454
+ -0.375,
3455
+ -0.375,
3456
+ -0.375,
3457
+ 0.0
3458
+ ],
3459
+ "q01": [
3460
+ -0.4285714328289032,
3461
+ -0.9800000190734863,
3462
+ -0.5571428537368774,
3463
+ -0.375,
3464
+ -0.15642857551574707,
3465
+ -0.335357129573822,
3466
+ 0.0
3467
+ ],
3468
+ "q99": [
3469
+ 0.5914285778999329,
3470
+ 0.9714285731315613,
3471
+ 1.0,
3472
+ 0.3278571367263794,
3473
+ 0.207857146859169,
3474
+ 0.25607141852378845,
3475
+ 1.0
3476
+ ],
3477
+ "std": [
3478
+ 0.1875014752149582,
3479
+ 0.4468473494052887,
3480
+ 0.3792876601219177,
3481
+ 0.14097853004932404,
3482
+ 0.06453701853752136,
3483
+ 0.11765272170305252,
3484
+ 0.501045286655426
3485
+ ]
3486
+ },
3487
+ "num_trajectories": 1500,
3488
+ "num_transitions": 361883,
3489
+ "proprio": {
3490
+ "max": [
3491
+ 0.0,
3492
+ 0.0,
3493
+ 0.0,
3494
+ 0.0,
3495
+ 0.0,
3496
+ 0.0,
3497
+ 0.0
3498
+ ],
3499
+ "mean": [
3500
+ 0.0,
3501
+ 0.0,
3502
+ 0.0,
3503
+ 0.0,
3504
+ 0.0,
3505
+ 0.0,
3506
+ 0.0
3507
+ ],
3508
+ "min": [
3509
+ 0.0,
3510
+ 0.0,
3511
+ 0.0,
3512
+ 0.0,
3513
+ 0.0,
3514
+ 0.0,
3515
+ 0.0
3516
+ ],
3517
+ "q01": [
3518
+ 0.0,
3519
+ 0.0,
3520
+ 0.0,
3521
+ 0.0,
3522
+ 0.0,
3523
+ 0.0,
3524
+ 0.0
3525
+ ],
3526
+ "q99": [
3527
+ 0.0,
3528
+ 0.0,
3529
+ 0.0,
3530
+ 0.0,
3531
+ 0.0,
3532
+ 0.0,
3533
+ 0.0
3534
+ ],
3535
+ "std": [
3536
+ 0.0,
3537
+ 0.0,
3538
+ 0.0,
3539
+ 0.0,
3540
+ 0.0,
3541
+ 0.0,
3542
+ 0.0
3543
+ ]
3544
+ }
3545
+ },
3546
+ "viola": {
3547
+ "action": {
3548
+ "mask": [
3549
+ true,
3550
+ true,
3551
+ true,
3552
+ true,
3553
+ true,
3554
+ true,
3555
+ false
3556
+ ],
3557
+ "max": [
3558
+ 1.0,
3559
+ 1.0,
3560
+ 1.0,
3561
+ 0.375,
3562
+ 0.36321428418159485,
3563
+ 0.375,
3564
+ 1.0
3565
+ ],
3566
+ "mean": [
3567
+ 0.04761844128370285,
3568
+ -0.029204415157437325,
3569
+ 0.05586736649274826,
3570
+ -0.002618510741740465,
3571
+ 0.006867344491183758,
3572
+ -0.01682133786380291,
3573
+ 0.7323777675628662
3574
+ ],
3575
+ "min": [
3576
+ -1.0,
3577
+ -1.0,
3578
+ -1.0,
3579
+ -0.375,
3580
+ -0.375,
3581
+ -0.375,
3582
+ 0.0
3583
+ ],
3584
+ "q01": [
3585
+ -0.9628571271896362,
3586
+ -1.0,
3587
+ -1.0,
3588
+ -0.26249998807907104,
3589
+ -0.21321429312229156,
3590
+ -0.3385714292526245,
3591
+ 0.0
3592
+ ],
3593
+ "q99": [
3594
+ 0.9114285707473755,
3595
+ 0.868571400642395,
3596
+ 1.0,
3597
+ 0.2817857265472412,
3598
+ 0.2239285707473755,
3599
+ 0.3557142913341522,
3600
+ 1.0
3601
+ ],
3602
+ "std": [
3603
+ 0.39157867431640625,
3604
+ 0.4076525568962097,
3605
+ 0.40077948570251465,
3606
+ 0.10023996233940125,
3607
+ 0.0844319611787796,
3608
+ 0.10375042259693146,
3609
+ 0.44260647892951965
3610
+ ]
3611
+ },
3612
+ "num_trajectories": 150,
3613
+ "num_transitions": 76324,
3614
+ "proprio": {
3615
+ "max": [
3616
+ 0.0,
3617
+ 0.0,
3618
+ 0.0,
3619
+ 0.0,
3620
+ 0.0,
3621
+ 0.0,
3622
+ 0.0
3623
+ ],
3624
+ "mean": [
3625
+ 0.0,
3626
+ 0.0,
3627
+ 0.0,
3628
+ 0.0,
3629
+ 0.0,
3630
+ 0.0,
3631
+ 0.0
3632
+ ],
3633
+ "min": [
3634
+ 0.0,
3635
+ 0.0,
3636
+ 0.0,
3637
+ 0.0,
3638
+ 0.0,
3639
+ 0.0,
3640
+ 0.0
3641
+ ],
3642
+ "q01": [
3643
+ 0.0,
3644
+ 0.0,
3645
+ 0.0,
3646
+ 0.0,
3647
+ 0.0,
3648
+ 0.0,
3649
+ 0.0
3650
+ ],
3651
+ "q99": [
3652
+ 0.0,
3653
+ 0.0,
3654
+ 0.0,
3655
+ 0.0,
3656
+ 0.0,
3657
+ 0.0,
3658
+ 0.0
3659
+ ],
3660
+ "std": [
3661
+ 0.0,
3662
+ 0.0,
3663
+ 0.0,
3664
+ 0.0,
3665
+ 0.0,
3666
+ 0.0,
3667
+ 0.0
3668
+ ]
3669
+ }
3670
+ }
3671
+ },
3672
+ "output_projector_states": false,
3673
+ "pad_to_multiple_of": 64,
3674
+ "pad_token_id": 32000,
3675
+ "text_config": {
3676
+ "model_type": "llama",
3677
+ "pad_token_id": 32000,
3678
+ "torch_dtype": "bfloat16",
3679
+ "vocab_size": 32064
3680
+ },
3681
+ "timm_model_ids": [
3682
+ "vit_large_patch14_reg4_dinov2.lvd142m",
3683
+ "vit_so400m_patch14_siglip_224"
3684
+ ],
3685
+ "timm_override_act_layers": [
3686
+ null,
3687
+ null
3688
+ ],
3689
+ "torch_dtype": "bfloat16",
3690
+ "transformers_version": "4.40.1",
3691
+ "use_fused_vision_backbone": true,
3692
+ "vision_backbone_id": "dinosiglip-vit-so-224px"
3693
+ }
configuration_prismatic.py ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ configuration_prismatic.py
3
+
4
+ HuggingFace-style configuration definition for Prismatic VLMs, inheriting from `transformers.PretrainedConfig`.
5
+ Default configuration specifies `siglip-224px+7b`.
6
+ """
7
+
8
+ from typing import Any, Dict, List, Optional
9
+
10
+ from transformers import PretrainedConfig
11
+ from transformers.models.auto import CONFIG_MAPPING
12
+
13
+ # === Utilities for Mapping Prismatic names to HF names ===
14
+ # fmt: off
15
+ VISION_BACKBONE_TO_RESOLUTION: Dict[str, List[int]] = {
16
+ "clip-vit-l": [224], "siglip-vit-so400m": [224], "dinov2-vit-l": [224], "in1k-vit-l": [224],
17
+
18
+ "clip-vit-l-336px": [336],
19
+ "siglip-vit-so400m-384px": [384],
20
+
21
+ "dinoclip-vit-l-336px": [336, 336],
22
+ "dinosiglip-vit-so-224px": [224, 224],
23
+ "dinosiglip-vit-so-384px": [384, 384],
24
+ }
25
+ VISION_BACKBONE_TO_TIMM_ID: Dict[str, List[str]] = {
26
+ "clip-vit-l": ["vit_large_patch14_clip_224.openai"],
27
+ "clip-vit-l-336px": ["vit_large_patch14_clip_336.openai"],
28
+
29
+ "dinov2-vit-l": ["vit_large_patch14_reg4_dinov2.lvd142m"],
30
+ "in1k-vit-l": ["vit_large_patch16_224.augreg_in21k_ft_in1k"],
31
+
32
+ "siglip-vit-so400m": ["vit_so400m_patch14_siglip_224"],
33
+ "siglip-vit-so400m-384px": ["vit_so400m_patch14_siglip_384"],
34
+
35
+ "dinoclip-vit-l-336px": ["vit_large_patch14_reg4_dinov2.lvd142m", "vit_large_patch14_clip_336.openai"],
36
+ "dinosiglip-vit-so-224px": ["vit_large_patch14_reg4_dinov2.lvd142m", "vit_so400m_patch14_siglip_224"],
37
+ "dinosiglip-vit-so-384px": ["vit_large_patch14_reg4_dinov2.lvd142m", "vit_so400m_patch14_siglip_384"],
38
+ }
39
+ TIMM_OVERRIDE_ACT_LAYER: Dict[str, List[Optional[str]]] = {
40
+ "clip-vit-l": ["quick_gelu"], "clip-vit-l-336px": ["quick_gelu"],
41
+ "dinov2-vit-l": [None], "in1k-vit-l": [None],
42
+ "siglip-vit-so400m": [None], "siglip-vit-so400m-384px": [None],
43
+ "dinoclip-vit-l-336px": [None, "quick_gelu"],
44
+ "dinosiglip-vit-so-224px": [None, None], "dinosiglip-vit-so-384px": [None, None]
45
+ }
46
+
47
+ LLM_BACKBONE_TO_HF_PATH = {
48
+ "llama2-7b-pure": "meta-llama/Llama-2-7b-hf", "llama2-13b-pure": "meta-llama/Llama-2-13b-hf",
49
+ "llama2-7b-chat": "meta-llama/Llama-2-7b-chat-hf", "llama2-13b-chat": "meta-llama/Llama-2-13b-chat-hf",
50
+
51
+ "vicuna-v15-7b": "lmsys/vicuna-7b-v1.5", "vicuna-v15-13b": "lmsys/vicuna-13b-v1.5",
52
+
53
+ "mistral-v0.1-7b-pure": "mistralai/Mistral-7B-v0.1",
54
+ "mistral-v0.1-7b-instruct": "mistralai/Mistral-7B-Instruct-v0.1",
55
+
56
+ "phi-2-3b": "microsoft/phi-2",
57
+ }
58
+ LLM_BACKBONE_TO_HF_METACLASS = {
59
+ "llama2-7b-pure": "llama", "llama2-13b-pure": "llama", "llama2-7b-chat": "llama", "llama2-13b-chat": "llama",
60
+ "vicuna-v15-7b": "llama", "vicuna-v15-13b": "llama",
61
+
62
+ "mistral-v0.1-7b-pure": "mistral", "mistral-v0.1-7b-instruct": "mistral",
63
+
64
+ "phi-2-3b": "phi",
65
+ }
66
+
67
+ VALID_VISION_BACKBONES = set(VISION_BACKBONE_TO_RESOLUTION.keys())
68
+ VALID_LLM_BACKBONES = set(LLM_BACKBONE_TO_HF_PATH)
69
+ # fmt: on
70
+
71
+
72
+ class PrismaticConfig(PretrainedConfig):
73
+ model_type: str = "prismatic"
74
+ is_composition: bool = False
75
+
76
+ def __init__(
77
+ self,
78
+ vision_backbone_id: str = "siglip-vit-so400m",
79
+ llm_backbone_id: str = "vicuna-v15-7b",
80
+ arch_specifier: str = "no-align+gelu-mlp",
81
+ use_fused_vision_backbone: Optional[bool] = None,
82
+ image_resize_strategy: str = "letterbox",
83
+ text_config: Optional[Dict[str, Any]] = None,
84
+ llm_max_length: int = 2048,
85
+ pad_token_id: int = 32000,
86
+ pad_to_multiple_of: int = 64,
87
+ output_projector_states: bool = False,
88
+ **kwargs: str,
89
+ ) -> None:
90
+ if vision_backbone_id not in VALID_VISION_BACKBONES:
91
+ raise ValueError(f"Vision backbone `{vision_backbone_id}` not in {VALID_VISION_BACKBONES = }")
92
+
93
+ if llm_backbone_id not in VALID_LLM_BACKBONES:
94
+ raise ValueError(f"LLM backbone `{llm_backbone_id}` not in {VALID_LLM_BACKBONES = }")
95
+
96
+ # Set Prismatic Configuration Fields
97
+ self.vision_backbone_id = vision_backbone_id
98
+ self.llm_backbone_id = llm_backbone_id
99
+ self.arch_specifier = arch_specifier
100
+ self.output_projector_states = output_projector_states
101
+
102
+ # [Contract] All vision backbone parameters are lists =>> supports fused backbones with different preprocessing
103
+ self.use_fused_vision_backbone = (
104
+ use_fused_vision_backbone
105
+ if use_fused_vision_backbone is not None
106
+ else any(self.vision_backbone_id.startswith(v) for v in ["dinoclip", "dinosiglip"])
107
+ )
108
+
109
+ self.timm_model_ids = VISION_BACKBONE_TO_TIMM_ID[self.vision_backbone_id]
110
+ self.timm_override_act_layers = TIMM_OVERRIDE_ACT_LAYER[self.vision_backbone_id]
111
+ self.image_sizes = VISION_BACKBONE_TO_RESOLUTION[self.vision_backbone_id]
112
+ self.image_resize_strategy = image_resize_strategy
113
+
114
+ self.hf_llm_id = LLM_BACKBONE_TO_HF_PATH[self.llm_backbone_id]
115
+ self.llm_max_length = llm_max_length
116
+ self.pad_token_id, self.pad_to_multiple_of = pad_token_id, pad_to_multiple_of
117
+
118
+ # [IMPORTANT] HF Utilities actually look for a `text_config` field... we need to use that specific naming!
119
+ self.text_config = (
120
+ CONFIG_MAPPING[LLM_BACKBONE_TO_HF_METACLASS[self.llm_backbone_id]](**text_config)
121
+ if text_config is not None
122
+ else CONFIG_MAPPING[LLM_BACKBONE_TO_HF_METACLASS[self.llm_backbone_id]]()
123
+ )
124
+
125
+ # Dispatch **kwargs to super() =>> note that `pad_token_id` collides, so we pass it in here as well...
126
+ super().__init__(pad_token_id=pad_token_id, **kwargs)
127
+
128
+
129
+ class OpenVLAConfig(PrismaticConfig):
130
+ model_type: str = "openvla"
131
+
132
+ def __init__(
133
+ self,
134
+ norm_stats: Optional[Dict[str, Dict[str, Dict[str, Dict[str, List[float]]]]]] = None,
135
+ n_action_bins: int = 256,
136
+ **kwargs: str,
137
+ ) -> None:
138
+ self.norm_stats, self.n_action_bins = norm_stats, n_action_bins
139
+
140
+ super().__init__(**kwargs)
dataset_statistics.json ADDED
@@ -0,0 +1,526 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "libero_10": {
3
+ "action": {
4
+ "mean": [
5
+ 0.06630383431911469,
6
+ 0.090694859623909,
7
+ -0.09185197949409485,
8
+ 0.00014266508514992893,
9
+ 0.005637437105178833,
10
+ -0.0038363179191946983,
11
+ 0.5176114439964294
12
+ ],
13
+ "std": [
14
+ 0.33593615889549255,
15
+ 0.3820026218891144,
16
+ 0.4535250663757324,
17
+ 0.03832726553082466,
18
+ 0.06335575133562088,
19
+ 0.07599934190511703,
20
+ 0.4993402063846588
21
+ ],
22
+ "max": [
23
+ 0.9375,
24
+ 0.9375,
25
+ 0.9375,
26
+ 0.3557142913341522,
27
+ 0.375,
28
+ 0.375,
29
+ 1.0
30
+ ],
31
+ "min": [
32
+ -0.9375,
33
+ -0.9375,
34
+ -0.9375,
35
+ -0.2582142949104309,
36
+ -0.375,
37
+ -0.3642857074737549,
38
+ 0.0
39
+ ],
40
+ "q01": [
41
+ -0.6883928775787354,
42
+ -0.8089285492897034,
43
+ -0.9375,
44
+ -0.1128571406006813,
45
+ -0.1639285683631897,
46
+ -0.20357142388820648,
47
+ 0.0
48
+ ],
49
+ "q99": [
50
+ 0.9375,
51
+ 0.8705357313156128,
52
+ 0.9375,
53
+ 0.12428571283817291,
54
+ 0.19285714626312256,
55
+ 0.3449999988079071,
56
+ 1.0
57
+ ],
58
+ "mask": [
59
+ true,
60
+ true,
61
+ true,
62
+ true,
63
+ true,
64
+ true,
65
+ false
66
+ ]
67
+ },
68
+ "proprio": {
69
+ "mean": [
70
+ -0.04460641369223595,
71
+ 0.035552240908145905,
72
+ 0.7428034543991089,
73
+ 2.9607746601104736,
74
+ -0.13809093832969666,
75
+ -0.10721249878406525,
76
+ 0.02668767236173153,
77
+ -0.02694464474916458
78
+ ],
79
+ "std": [
80
+ 0.10515860468149185,
81
+ 0.1530338078737259,
82
+ 0.38418349623680115,
83
+ 0.3309749960899353,
84
+ 0.8265467882156372,
85
+ 0.311494916677475,
86
+ 0.014123727567493916,
87
+ 0.014056944288313389
88
+ ],
89
+ "max": [
90
+ 0.21031762659549713,
91
+ 0.39128610491752625,
92
+ 1.3660105466842651,
93
+ 3.7743608951568604,
94
+ 3.5752453804016113,
95
+ 1.3747878074645996,
96
+ 0.042380549013614655,
97
+ 0.0010137537028640509
98
+ ],
99
+ "min": [
100
+ -0.4856736361980438,
101
+ -0.3313629925251007,
102
+ 0.008128180168569088,
103
+ 0.8415665030479431,
104
+ -3.616978406906128,
105
+ -1.842738389968872,
106
+ -0.002542249858379364,
107
+ -0.042119648307561874
108
+ ],
109
+ "q01": [
110
+ -0.40087422728538513,
111
+ -0.2639563336968422,
112
+ 0.029872100800275803,
113
+ 1.5078086853027344,
114
+ -2.591851234436035,
115
+ -1.0244591236114502,
116
+ 0.0017080969410017133,
117
+ -0.04002029821276665
118
+ ],
119
+ "q99": [
120
+ 0.13566908240318298,
121
+ 0.335424880683422,
122
+ 1.2716039419174194,
123
+ 3.2693305015563965,
124
+ 2.4190170764923096,
125
+ 0.6207390427589417,
126
+ 0.04029817134141922,
127
+ -0.0017327156383544207
128
+ ]
129
+ },
130
+ "num_transitions": 2238036,
131
+ "num_trajectories": 14347
132
+ },
133
+ "libero_object": {
134
+ "action": {
135
+ "mean": [
136
+ 0.06630383431911469,
137
+ 0.090694859623909,
138
+ -0.09185197949409485,
139
+ 0.00014266508514992893,
140
+ 0.005637437105178833,
141
+ -0.0038363179191946983,
142
+ 0.5176114439964294
143
+ ],
144
+ "std": [
145
+ 0.33593615889549255,
146
+ 0.3820026218891144,
147
+ 0.4535250663757324,
148
+ 0.03832726553082466,
149
+ 0.06335575133562088,
150
+ 0.07599934190511703,
151
+ 0.4993402063846588
152
+ ],
153
+ "max": [
154
+ 0.9375,
155
+ 0.9375,
156
+ 0.9375,
157
+ 0.3557142913341522,
158
+ 0.375,
159
+ 0.375,
160
+ 1.0
161
+ ],
162
+ "min": [
163
+ -0.9375,
164
+ -0.9375,
165
+ -0.9375,
166
+ -0.2582142949104309,
167
+ -0.375,
168
+ -0.3642857074737549,
169
+ 0.0
170
+ ],
171
+ "q01": [
172
+ -0.6883928775787354,
173
+ -0.8089285492897034,
174
+ -0.9375,
175
+ -0.1128571406006813,
176
+ -0.1639285683631897,
177
+ -0.20357142388820648,
178
+ 0.0
179
+ ],
180
+ "q99": [
181
+ 0.9375,
182
+ 0.8705357313156128,
183
+ 0.9375,
184
+ 0.12428571283817291,
185
+ 0.19285714626312256,
186
+ 0.3449999988079071,
187
+ 1.0
188
+ ],
189
+ "mask": [
190
+ true,
191
+ true,
192
+ true,
193
+ true,
194
+ true,
195
+ true,
196
+ false
197
+ ]
198
+ },
199
+ "proprio": {
200
+ "mean": [
201
+ -0.04460641369223595,
202
+ 0.035552240908145905,
203
+ 0.7428034543991089,
204
+ 2.9607746601104736,
205
+ -0.13809093832969666,
206
+ -0.10721249878406525,
207
+ 0.02668767236173153,
208
+ -0.02694464474916458
209
+ ],
210
+ "std": [
211
+ 0.10515860468149185,
212
+ 0.1530338078737259,
213
+ 0.38418349623680115,
214
+ 0.3309749960899353,
215
+ 0.8265467882156372,
216
+ 0.311494916677475,
217
+ 0.014123727567493916,
218
+ 0.014056944288313389
219
+ ],
220
+ "max": [
221
+ 0.21031762659549713,
222
+ 0.39128610491752625,
223
+ 1.3660105466842651,
224
+ 3.7743608951568604,
225
+ 3.5752453804016113,
226
+ 1.3747878074645996,
227
+ 0.042380549013614655,
228
+ 0.0010137537028640509
229
+ ],
230
+ "min": [
231
+ -0.4856736361980438,
232
+ -0.3313629925251007,
233
+ 0.008128180168569088,
234
+ 0.8415665030479431,
235
+ -3.616978406906128,
236
+ -1.842738389968872,
237
+ -0.002542249858379364,
238
+ -0.042119648307561874
239
+ ],
240
+ "q01": [
241
+ -0.40087422728538513,
242
+ -0.2639563336968422,
243
+ 0.029872100800275803,
244
+ 1.5078086853027344,
245
+ -2.591851234436035,
246
+ -1.0244591236114502,
247
+ 0.0017080969410017133,
248
+ -0.04002029821276665
249
+ ],
250
+ "q99": [
251
+ 0.13566908240318298,
252
+ 0.335424880683422,
253
+ 1.2716039419174194,
254
+ 3.2693305015563965,
255
+ 2.4190170764923096,
256
+ 0.6207390427589417,
257
+ 0.04029817134141922,
258
+ -0.0017327156383544207
259
+ ]
260
+ },
261
+ "num_transitions": 2238036,
262
+ "num_trajectories": 14347
263
+ },
264
+ "libero_goal": {
265
+ "action": {
266
+ "mean": [
267
+ 0.06630383431911469,
268
+ 0.090694859623909,
269
+ -0.09185197949409485,
270
+ 0.00014266508514992893,
271
+ 0.005637437105178833,
272
+ -0.0038363179191946983,
273
+ 0.5176114439964294
274
+ ],
275
+ "std": [
276
+ 0.33593615889549255,
277
+ 0.3820026218891144,
278
+ 0.4535250663757324,
279
+ 0.03832726553082466,
280
+ 0.06335575133562088,
281
+ 0.07599934190511703,
282
+ 0.4993402063846588
283
+ ],
284
+ "max": [
285
+ 0.9375,
286
+ 0.9375,
287
+ 0.9375,
288
+ 0.3557142913341522,
289
+ 0.375,
290
+ 0.375,
291
+ 1.0
292
+ ],
293
+ "min": [
294
+ -0.9375,
295
+ -0.9375,
296
+ -0.9375,
297
+ -0.2582142949104309,
298
+ -0.375,
299
+ -0.3642857074737549,
300
+ 0.0
301
+ ],
302
+ "q01": [
303
+ -0.6883928775787354,
304
+ -0.8089285492897034,
305
+ -0.9375,
306
+ -0.1128571406006813,
307
+ -0.1639285683631897,
308
+ -0.20357142388820648,
309
+ 0.0
310
+ ],
311
+ "q99": [
312
+ 0.9375,
313
+ 0.8705357313156128,
314
+ 0.9375,
315
+ 0.12428571283817291,
316
+ 0.19285714626312256,
317
+ 0.3449999988079071,
318
+ 1.0
319
+ ],
320
+ "mask": [
321
+ true,
322
+ true,
323
+ true,
324
+ true,
325
+ true,
326
+ true,
327
+ false
328
+ ]
329
+ },
330
+ "proprio": {
331
+ "mean": [
332
+ -0.04460641369223595,
333
+ 0.035552240908145905,
334
+ 0.7428034543991089,
335
+ 2.9607746601104736,
336
+ -0.13809093832969666,
337
+ -0.10721249878406525,
338
+ 0.02668767236173153,
339
+ -0.02694464474916458
340
+ ],
341
+ "std": [
342
+ 0.10515860468149185,
343
+ 0.1530338078737259,
344
+ 0.38418349623680115,
345
+ 0.3309749960899353,
346
+ 0.8265467882156372,
347
+ 0.311494916677475,
348
+ 0.014123727567493916,
349
+ 0.014056944288313389
350
+ ],
351
+ "max": [
352
+ 0.21031762659549713,
353
+ 0.39128610491752625,
354
+ 1.3660105466842651,
355
+ 3.7743608951568604,
356
+ 3.5752453804016113,
357
+ 1.3747878074645996,
358
+ 0.042380549013614655,
359
+ 0.0010137537028640509
360
+ ],
361
+ "min": [
362
+ -0.4856736361980438,
363
+ -0.3313629925251007,
364
+ 0.008128180168569088,
365
+ 0.8415665030479431,
366
+ -3.616978406906128,
367
+ -1.842738389968872,
368
+ -0.002542249858379364,
369
+ -0.042119648307561874
370
+ ],
371
+ "q01": [
372
+ -0.40087422728538513,
373
+ -0.2639563336968422,
374
+ 0.029872100800275803,
375
+ 1.5078086853027344,
376
+ -2.591851234436035,
377
+ -1.0244591236114502,
378
+ 0.0017080969410017133,
379
+ -0.04002029821276665
380
+ ],
381
+ "q99": [
382
+ 0.13566908240318298,
383
+ 0.335424880683422,
384
+ 1.2716039419174194,
385
+ 3.2693305015563965,
386
+ 2.4190170764923096,
387
+ 0.6207390427589417,
388
+ 0.04029817134141922,
389
+ -0.0017327156383544207
390
+ ]
391
+ },
392
+ "num_transitions": 2238036,
393
+ "num_trajectories": 14347
394
+ },
395
+ "libero_spatial": {
396
+ "action": {
397
+ "mean": [
398
+ 0.06630383431911469,
399
+ 0.090694859623909,
400
+ -0.09185197949409485,
401
+ 0.00014266508514992893,
402
+ 0.005637437105178833,
403
+ -0.0038363179191946983,
404
+ 0.5176114439964294
405
+ ],
406
+ "std": [
407
+ 0.33593615889549255,
408
+ 0.3820026218891144,
409
+ 0.4535250663757324,
410
+ 0.03832726553082466,
411
+ 0.06335575133562088,
412
+ 0.07599934190511703,
413
+ 0.4993402063846588
414
+ ],
415
+ "max": [
416
+ 0.9375,
417
+ 0.9375,
418
+ 0.9375,
419
+ 0.3557142913341522,
420
+ 0.375,
421
+ 0.375,
422
+ 1.0
423
+ ],
424
+ "min": [
425
+ -0.9375,
426
+ -0.9375,
427
+ -0.9375,
428
+ -0.2582142949104309,
429
+ -0.375,
430
+ -0.3642857074737549,
431
+ 0.0
432
+ ],
433
+ "q01": [
434
+ -0.6883928775787354,
435
+ -0.8089285492897034,
436
+ -0.9375,
437
+ -0.1128571406006813,
438
+ -0.1639285683631897,
439
+ -0.20357142388820648,
440
+ 0.0
441
+ ],
442
+ "q99": [
443
+ 0.9375,
444
+ 0.8705357313156128,
445
+ 0.9375,
446
+ 0.12428571283817291,
447
+ 0.19285714626312256,
448
+ 0.3449999988079071,
449
+ 1.0
450
+ ],
451
+ "mask": [
452
+ true,
453
+ true,
454
+ true,
455
+ true,
456
+ true,
457
+ true,
458
+ false
459
+ ]
460
+ },
461
+ "proprio": {
462
+ "mean": [
463
+ -0.04460641369223595,
464
+ 0.035552240908145905,
465
+ 0.7428034543991089,
466
+ 2.9607746601104736,
467
+ -0.13809093832969666,
468
+ -0.10721249878406525,
469
+ 0.02668767236173153,
470
+ -0.02694464474916458
471
+ ],
472
+ "std": [
473
+ 0.10515860468149185,
474
+ 0.1530338078737259,
475
+ 0.38418349623680115,
476
+ 0.3309749960899353,
477
+ 0.8265467882156372,
478
+ 0.311494916677475,
479
+ 0.014123727567493916,
480
+ 0.014056944288313389
481
+ ],
482
+ "max": [
483
+ 0.21031762659549713,
484
+ 0.39128610491752625,
485
+ 1.3660105466842651,
486
+ 3.7743608951568604,
487
+ 3.5752453804016113,
488
+ 1.3747878074645996,
489
+ 0.042380549013614655,
490
+ 0.0010137537028640509
491
+ ],
492
+ "min": [
493
+ -0.4856736361980438,
494
+ -0.3313629925251007,
495
+ 0.008128180168569088,
496
+ 0.8415665030479431,
497
+ -3.616978406906128,
498
+ -1.842738389968872,
499
+ -0.002542249858379364,
500
+ -0.042119648307561874
501
+ ],
502
+ "q01": [
503
+ -0.40087422728538513,
504
+ -0.2639563336968422,
505
+ 0.029872100800275803,
506
+ 1.5078086853027344,
507
+ -2.591851234436035,
508
+ -1.0244591236114502,
509
+ 0.0017080969410017133,
510
+ -0.04002029821276665
511
+ ],
512
+ "q99": [
513
+ 0.13566908240318298,
514
+ 0.335424880683422,
515
+ 1.2716039419174194,
516
+ 3.2693305015563965,
517
+ 2.4190170764923096,
518
+ 0.6207390427589417,
519
+ 0.04029817134141922,
520
+ -0.0017327156383544207
521
+ ]
522
+ },
523
+ "num_transitions": 2238036,
524
+ "num_trajectories": 14347
525
+ }
526
+ }
generation_config.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 1,
4
+ "eos_token_id": 2,
5
+ "pad_token_id": 32000,
6
+ "transformers_version": "4.40.1"
7
+ }
lora_adapter/README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: moojink/openvla-7b-oft-finetuned-libero-spatial-object-goal-10
3
+ library_name: peft
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.11.1
lora_adapter/adapter_config.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": {
4
+ "base_model_class": "OpenVLAForActionPrediction",
5
+ "parent_library": "transformers_modules.openvla-7b-oft-finetuned-libero-spatial-object-goal-10.modeling_prismatic"
6
+ },
7
+ "base_model_name_or_path": "/inspire/hdd/project/embodied-multimodality/public/syfei/torch/hub/openvla-7b-oft-finetuned-libero-spatial-object-goal-10",
8
+ "bias": "none",
9
+ "fan_in_fan_out": false,
10
+ "inference_mode": true,
11
+ "init_lora_weights": "gaussian",
12
+ "layer_replication": null,
13
+ "layers_pattern": null,
14
+ "layers_to_transform": null,
15
+ "loftq_config": {},
16
+ "lora_alpha": 16,
17
+ "lora_dropout": 0.0,
18
+ "megatron_config": null,
19
+ "megatron_core": "megatron.core",
20
+ "modules_to_save": null,
21
+ "peft_type": "LORA",
22
+ "r": 32,
23
+ "rank_pattern": {},
24
+ "revision": null,
25
+ "target_modules": [
26
+ "down_proj",
27
+ "q_proj",
28
+ "lm_head",
29
+ "v_proj",
30
+ "proj",
31
+ "fc2",
32
+ "q",
33
+ "fc3",
34
+ "qkv",
35
+ "k_proj",
36
+ "o_proj",
37
+ "up_proj",
38
+ "fc1",
39
+ "kv",
40
+ "gate_proj"
41
+ ],
42
+ "task_type": null,
43
+ "use_dora": false,
44
+ "use_rslora": false
45
+ }
lora_adapter/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dbd9440767b9b70994f9f0a6bd67dfc250f4ec37649ec2e10cdf34f46e1c9244
3
+ size 484458600
model-00001-of-00004.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:45f48b9bc6d7535f5498e826b2b0d9da4aae03853bd4681a10decb480de55fd3
3
+ size 4925122448
model-00002-of-00004.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:47500ca78f427bfb675879d93bea6f514480f460740610458addef1921c4522b
3
+ size 4947392496
model-00003-of-00004.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17cc127c4ed727708a6138103392b79290d45bb8c9edac949674625b0d2d0bf5
3
+ size 4947417456
model-00004-of-00004.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a877e3fece1feafb80f59f91585ce04379ee39e2bf9a25cb7b4acf237e896e60
3
+ size 262668432
model.safetensors.index.json ADDED
@@ -0,0 +1,989 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "total_size": 15082474368
4
+ },
5
+ "weight_map": {
6
+ "language_model.lm_head.weight": "model-00004-of-00004.safetensors",
7
+ "language_model.model.embed_tokens.weight": "model-00001-of-00004.safetensors",
8
+ "language_model.model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
9
+ "language_model.model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
10
+ "language_model.model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
11
+ "language_model.model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
12
+ "language_model.model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
13
+ "language_model.model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
14
+ "language_model.model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
15
+ "language_model.model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
16
+ "language_model.model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
17
+ "language_model.model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
18
+ "language_model.model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
19
+ "language_model.model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
20
+ "language_model.model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
21
+ "language_model.model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
22
+ "language_model.model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
23
+ "language_model.model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
24
+ "language_model.model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
25
+ "language_model.model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
26
+ "language_model.model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
27
+ "language_model.model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
28
+ "language_model.model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
29
+ "language_model.model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
30
+ "language_model.model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
31
+ "language_model.model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
32
+ "language_model.model.layers.10.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
33
+ "language_model.model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
34
+ "language_model.model.layers.10.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
35
+ "language_model.model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
36
+ "language_model.model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
37
+ "language_model.model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
38
+ "language_model.model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
39
+ "language_model.model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
40
+ "language_model.model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
41
+ "language_model.model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
42
+ "language_model.model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
43
+ "language_model.model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
44
+ "language_model.model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
45
+ "language_model.model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
46
+ "language_model.model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
47
+ "language_model.model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
48
+ "language_model.model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
49
+ "language_model.model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
50
+ "language_model.model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
51
+ "language_model.model.layers.12.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
52
+ "language_model.model.layers.12.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
53
+ "language_model.model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
54
+ "language_model.model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
55
+ "language_model.model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
56
+ "language_model.model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
57
+ "language_model.model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
58
+ "language_model.model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
59
+ "language_model.model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
60
+ "language_model.model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
61
+ "language_model.model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
62
+ "language_model.model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
63
+ "language_model.model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
64
+ "language_model.model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
65
+ "language_model.model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
66
+ "language_model.model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
67
+ "language_model.model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
68
+ "language_model.model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
69
+ "language_model.model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
70
+ "language_model.model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
71
+ "language_model.model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
72
+ "language_model.model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
73
+ "language_model.model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
74
+ "language_model.model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
75
+ "language_model.model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
76
+ "language_model.model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
77
+ "language_model.model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
78
+ "language_model.model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
79
+ "language_model.model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
80
+ "language_model.model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors",
81
+ "language_model.model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
82
+ "language_model.model.layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
83
+ "language_model.model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
84
+ "language_model.model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
85
+ "language_model.model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
86
+ "language_model.model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
87
+ "language_model.model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
88
+ "language_model.model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
89
+ "language_model.model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors",
90
+ "language_model.model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
91
+ "language_model.model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
92
+ "language_model.model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
93
+ "language_model.model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
94
+ "language_model.model.layers.17.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
95
+ "language_model.model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
96
+ "language_model.model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
97
+ "language_model.model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
98
+ "language_model.model.layers.18.input_layernorm.weight": "model-00002-of-00004.safetensors",
99
+ "language_model.model.layers.18.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
100
+ "language_model.model.layers.18.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
101
+ "language_model.model.layers.18.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
102
+ "language_model.model.layers.18.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
103
+ "language_model.model.layers.18.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
104
+ "language_model.model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
105
+ "language_model.model.layers.18.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
106
+ "language_model.model.layers.18.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
107
+ "language_model.model.layers.19.input_layernorm.weight": "model-00003-of-00004.safetensors",
108
+ "language_model.model.layers.19.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
109
+ "language_model.model.layers.19.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
110
+ "language_model.model.layers.19.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
111
+ "language_model.model.layers.19.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
112
+ "language_model.model.layers.19.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
113
+ "language_model.model.layers.19.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
114
+ "language_model.model.layers.19.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
115
+ "language_model.model.layers.19.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
116
+ "language_model.model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
117
+ "language_model.model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
118
+ "language_model.model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
119
+ "language_model.model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
120
+ "language_model.model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
121
+ "language_model.model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
122
+ "language_model.model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
123
+ "language_model.model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
124
+ "language_model.model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
125
+ "language_model.model.layers.20.input_layernorm.weight": "model-00003-of-00004.safetensors",
126
+ "language_model.model.layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
127
+ "language_model.model.layers.20.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
128
+ "language_model.model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
129
+ "language_model.model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
130
+ "language_model.model.layers.20.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
131
+ "language_model.model.layers.20.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
132
+ "language_model.model.layers.20.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
133
+ "language_model.model.layers.20.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
134
+ "language_model.model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors",
135
+ "language_model.model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
136
+ "language_model.model.layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
137
+ "language_model.model.layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
138
+ "language_model.model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
139
+ "language_model.model.layers.21.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
140
+ "language_model.model.layers.21.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
141
+ "language_model.model.layers.21.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
142
+ "language_model.model.layers.21.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
143
+ "language_model.model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
144
+ "language_model.model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
145
+ "language_model.model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
146
+ "language_model.model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
147
+ "language_model.model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
148
+ "language_model.model.layers.22.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
149
+ "language_model.model.layers.22.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
150
+ "language_model.model.layers.22.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
151
+ "language_model.model.layers.22.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
152
+ "language_model.model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
153
+ "language_model.model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
154
+ "language_model.model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
155
+ "language_model.model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
156
+ "language_model.model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
157
+ "language_model.model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
158
+ "language_model.model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
159
+ "language_model.model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
160
+ "language_model.model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
161
+ "language_model.model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
162
+ "language_model.model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
163
+ "language_model.model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
164
+ "language_model.model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
165
+ "language_model.model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
166
+ "language_model.model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
167
+ "language_model.model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
168
+ "language_model.model.layers.24.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
169
+ "language_model.model.layers.24.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
170
+ "language_model.model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
171
+ "language_model.model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
172
+ "language_model.model.layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
173
+ "language_model.model.layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
174
+ "language_model.model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
175
+ "language_model.model.layers.25.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
176
+ "language_model.model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
177
+ "language_model.model.layers.25.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
178
+ "language_model.model.layers.25.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
179
+ "language_model.model.layers.26.input_layernorm.weight": "model-00003-of-00004.safetensors",
180
+ "language_model.model.layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
181
+ "language_model.model.layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
182
+ "language_model.model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
183
+ "language_model.model.layers.26.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
184
+ "language_model.model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
185
+ "language_model.model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
186
+ "language_model.model.layers.26.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
187
+ "language_model.model.layers.26.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
188
+ "language_model.model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors",
189
+ "language_model.model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
190
+ "language_model.model.layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
191
+ "language_model.model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
192
+ "language_model.model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
193
+ "language_model.model.layers.27.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
194
+ "language_model.model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
195
+ "language_model.model.layers.27.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
196
+ "language_model.model.layers.27.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
197
+ "language_model.model.layers.28.input_layernorm.weight": "model-00003-of-00004.safetensors",
198
+ "language_model.model.layers.28.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
199
+ "language_model.model.layers.28.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
200
+ "language_model.model.layers.28.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
201
+ "language_model.model.layers.28.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
202
+ "language_model.model.layers.28.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
203
+ "language_model.model.layers.28.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
204
+ "language_model.model.layers.28.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
205
+ "language_model.model.layers.28.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
206
+ "language_model.model.layers.29.input_layernorm.weight": "model-00003-of-00004.safetensors",
207
+ "language_model.model.layers.29.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
208
+ "language_model.model.layers.29.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
209
+ "language_model.model.layers.29.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
210
+ "language_model.model.layers.29.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
211
+ "language_model.model.layers.29.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
212
+ "language_model.model.layers.29.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
213
+ "language_model.model.layers.29.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
214
+ "language_model.model.layers.29.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
215
+ "language_model.model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
216
+ "language_model.model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
217
+ "language_model.model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
218
+ "language_model.model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
219
+ "language_model.model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
220
+ "language_model.model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
221
+ "language_model.model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
222
+ "language_model.model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
223
+ "language_model.model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
224
+ "language_model.model.layers.30.input_layernorm.weight": "model-00003-of-00004.safetensors",
225
+ "language_model.model.layers.30.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
226
+ "language_model.model.layers.30.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
227
+ "language_model.model.layers.30.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
228
+ "language_model.model.layers.30.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
229
+ "language_model.model.layers.30.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
230
+ "language_model.model.layers.30.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
231
+ "language_model.model.layers.30.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
232
+ "language_model.model.layers.30.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
233
+ "language_model.model.layers.31.input_layernorm.weight": "model-00003-of-00004.safetensors",
234
+ "language_model.model.layers.31.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
235
+ "language_model.model.layers.31.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
236
+ "language_model.model.layers.31.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
237
+ "language_model.model.layers.31.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
238
+ "language_model.model.layers.31.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
239
+ "language_model.model.layers.31.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
240
+ "language_model.model.layers.31.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
241
+ "language_model.model.layers.31.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
242
+ "language_model.model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
243
+ "language_model.model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
244
+ "language_model.model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
245
+ "language_model.model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
246
+ "language_model.model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
247
+ "language_model.model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
248
+ "language_model.model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
249
+ "language_model.model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
250
+ "language_model.model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
251
+ "language_model.model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
252
+ "language_model.model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
253
+ "language_model.model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
254
+ "language_model.model.layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
255
+ "language_model.model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
256
+ "language_model.model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
257
+ "language_model.model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
258
+ "language_model.model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
259
+ "language_model.model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
260
+ "language_model.model.layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors",
261
+ "language_model.model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
262
+ "language_model.model.layers.6.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
263
+ "language_model.model.layers.6.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
264
+ "language_model.model.layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
265
+ "language_model.model.layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
266
+ "language_model.model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
267
+ "language_model.model.layers.6.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
268
+ "language_model.model.layers.6.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
269
+ "language_model.model.layers.7.input_layernorm.weight": "model-00002-of-00004.safetensors",
270
+ "language_model.model.layers.7.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
271
+ "language_model.model.layers.7.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
272
+ "language_model.model.layers.7.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
273
+ "language_model.model.layers.7.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
274
+ "language_model.model.layers.7.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
275
+ "language_model.model.layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
276
+ "language_model.model.layers.7.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
277
+ "language_model.model.layers.7.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
278
+ "language_model.model.layers.8.input_layernorm.weight": "model-00002-of-00004.safetensors",
279
+ "language_model.model.layers.8.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
280
+ "language_model.model.layers.8.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
281
+ "language_model.model.layers.8.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
282
+ "language_model.model.layers.8.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
283
+ "language_model.model.layers.8.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
284
+ "language_model.model.layers.8.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
285
+ "language_model.model.layers.8.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
286
+ "language_model.model.layers.8.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
287
+ "language_model.model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
288
+ "language_model.model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
289
+ "language_model.model.layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
290
+ "language_model.model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
291
+ "language_model.model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
292
+ "language_model.model.layers.9.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
293
+ "language_model.model.layers.9.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
294
+ "language_model.model.layers.9.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
295
+ "language_model.model.layers.9.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
296
+ "language_model.model.norm.weight": "model-00003-of-00004.safetensors",
297
+ "projector.fc1.bias": "model-00001-of-00004.safetensors",
298
+ "projector.fc1.weight": "model-00001-of-00004.safetensors",
299
+ "projector.fc2.bias": "model-00001-of-00004.safetensors",
300
+ "projector.fc2.weight": "model-00001-of-00004.safetensors",
301
+ "projector.fc3.bias": "model-00001-of-00004.safetensors",
302
+ "projector.fc3.weight": "model-00001-of-00004.safetensors",
303
+ "vision_backbone.featurizer.blocks.0.attn.proj.bias": "model-00001-of-00004.safetensors",
304
+ "vision_backbone.featurizer.blocks.0.attn.proj.weight": "model-00001-of-00004.safetensors",
305
+ "vision_backbone.featurizer.blocks.0.attn.qkv.bias": "model-00001-of-00004.safetensors",
306
+ "vision_backbone.featurizer.blocks.0.attn.qkv.weight": "model-00001-of-00004.safetensors",
307
+ "vision_backbone.featurizer.blocks.0.ls1.scale_factor": "model-00001-of-00004.safetensors",
308
+ "vision_backbone.featurizer.blocks.0.ls2.scale_factor": "model-00001-of-00004.safetensors",
309
+ "vision_backbone.featurizer.blocks.0.mlp.fc1.bias": "model-00001-of-00004.safetensors",
310
+ "vision_backbone.featurizer.blocks.0.mlp.fc1.weight": "model-00001-of-00004.safetensors",
311
+ "vision_backbone.featurizer.blocks.0.mlp.fc2.bias": "model-00001-of-00004.safetensors",
312
+ "vision_backbone.featurizer.blocks.0.mlp.fc2.weight": "model-00001-of-00004.safetensors",
313
+ "vision_backbone.featurizer.blocks.0.norm1.bias": "model-00001-of-00004.safetensors",
314
+ "vision_backbone.featurizer.blocks.0.norm1.weight": "model-00001-of-00004.safetensors",
315
+ "vision_backbone.featurizer.blocks.0.norm2.bias": "model-00001-of-00004.safetensors",
316
+ "vision_backbone.featurizer.blocks.0.norm2.weight": "model-00001-of-00004.safetensors",
317
+ "vision_backbone.featurizer.blocks.1.attn.proj.bias": "model-00001-of-00004.safetensors",
318
+ "vision_backbone.featurizer.blocks.1.attn.proj.weight": "model-00001-of-00004.safetensors",
319
+ "vision_backbone.featurizer.blocks.1.attn.qkv.bias": "model-00001-of-00004.safetensors",
320
+ "vision_backbone.featurizer.blocks.1.attn.qkv.weight": "model-00001-of-00004.safetensors",
321
+ "vision_backbone.featurizer.blocks.1.ls1.scale_factor": "model-00001-of-00004.safetensors",
322
+ "vision_backbone.featurizer.blocks.1.ls2.scale_factor": "model-00001-of-00004.safetensors",
323
+ "vision_backbone.featurizer.blocks.1.mlp.fc1.bias": "model-00001-of-00004.safetensors",
324
+ "vision_backbone.featurizer.blocks.1.mlp.fc1.weight": "model-00001-of-00004.safetensors",
325
+ "vision_backbone.featurizer.blocks.1.mlp.fc2.bias": "model-00001-of-00004.safetensors",
326
+ "vision_backbone.featurizer.blocks.1.mlp.fc2.weight": "model-00001-of-00004.safetensors",
327
+ "vision_backbone.featurizer.blocks.1.norm1.bias": "model-00001-of-00004.safetensors",
328
+ "vision_backbone.featurizer.blocks.1.norm1.weight": "model-00001-of-00004.safetensors",
329
+ "vision_backbone.featurizer.blocks.1.norm2.bias": "model-00001-of-00004.safetensors",
330
+ "vision_backbone.featurizer.blocks.1.norm2.weight": "model-00001-of-00004.safetensors",
331
+ "vision_backbone.featurizer.blocks.10.attn.proj.bias": "model-00001-of-00004.safetensors",
332
+ "vision_backbone.featurizer.blocks.10.attn.proj.weight": "model-00001-of-00004.safetensors",
333
+ "vision_backbone.featurizer.blocks.10.attn.qkv.bias": "model-00001-of-00004.safetensors",
334
+ "vision_backbone.featurizer.blocks.10.attn.qkv.weight": "model-00001-of-00004.safetensors",
335
+ "vision_backbone.featurizer.blocks.10.ls1.scale_factor": "model-00001-of-00004.safetensors",
336
+ "vision_backbone.featurizer.blocks.10.ls2.scale_factor": "model-00001-of-00004.safetensors",
337
+ "vision_backbone.featurizer.blocks.10.mlp.fc1.bias": "model-00001-of-00004.safetensors",
338
+ "vision_backbone.featurizer.blocks.10.mlp.fc1.weight": "model-00001-of-00004.safetensors",
339
+ "vision_backbone.featurizer.blocks.10.mlp.fc2.bias": "model-00001-of-00004.safetensors",
340
+ "vision_backbone.featurizer.blocks.10.mlp.fc2.weight": "model-00001-of-00004.safetensors",
341
+ "vision_backbone.featurizer.blocks.10.norm1.bias": "model-00001-of-00004.safetensors",
342
+ "vision_backbone.featurizer.blocks.10.norm1.weight": "model-00001-of-00004.safetensors",
343
+ "vision_backbone.featurizer.blocks.10.norm2.bias": "model-00001-of-00004.safetensors",
344
+ "vision_backbone.featurizer.blocks.10.norm2.weight": "model-00001-of-00004.safetensors",
345
+ "vision_backbone.featurizer.blocks.11.attn.proj.bias": "model-00001-of-00004.safetensors",
346
+ "vision_backbone.featurizer.blocks.11.attn.proj.weight": "model-00001-of-00004.safetensors",
347
+ "vision_backbone.featurizer.blocks.11.attn.qkv.bias": "model-00001-of-00004.safetensors",
348
+ "vision_backbone.featurizer.blocks.11.attn.qkv.weight": "model-00001-of-00004.safetensors",
349
+ "vision_backbone.featurizer.blocks.11.ls1.scale_factor": "model-00001-of-00004.safetensors",
350
+ "vision_backbone.featurizer.blocks.11.ls2.scale_factor": "model-00001-of-00004.safetensors",
351
+ "vision_backbone.featurizer.blocks.11.mlp.fc1.bias": "model-00001-of-00004.safetensors",
352
+ "vision_backbone.featurizer.blocks.11.mlp.fc1.weight": "model-00001-of-00004.safetensors",
353
+ "vision_backbone.featurizer.blocks.11.mlp.fc2.bias": "model-00001-of-00004.safetensors",
354
+ "vision_backbone.featurizer.blocks.11.mlp.fc2.weight": "model-00001-of-00004.safetensors",
355
+ "vision_backbone.featurizer.blocks.11.norm1.bias": "model-00001-of-00004.safetensors",
356
+ "vision_backbone.featurizer.blocks.11.norm1.weight": "model-00001-of-00004.safetensors",
357
+ "vision_backbone.featurizer.blocks.11.norm2.bias": "model-00001-of-00004.safetensors",
358
+ "vision_backbone.featurizer.blocks.11.norm2.weight": "model-00001-of-00004.safetensors",
359
+ "vision_backbone.featurizer.blocks.12.attn.proj.bias": "model-00001-of-00004.safetensors",
360
+ "vision_backbone.featurizer.blocks.12.attn.proj.weight": "model-00001-of-00004.safetensors",
361
+ "vision_backbone.featurizer.blocks.12.attn.qkv.bias": "model-00001-of-00004.safetensors",
362
+ "vision_backbone.featurizer.blocks.12.attn.qkv.weight": "model-00001-of-00004.safetensors",
363
+ "vision_backbone.featurizer.blocks.12.ls1.scale_factor": "model-00001-of-00004.safetensors",
364
+ "vision_backbone.featurizer.blocks.12.ls2.scale_factor": "model-00001-of-00004.safetensors",
365
+ "vision_backbone.featurizer.blocks.12.mlp.fc1.bias": "model-00001-of-00004.safetensors",
366
+ "vision_backbone.featurizer.blocks.12.mlp.fc1.weight": "model-00001-of-00004.safetensors",
367
+ "vision_backbone.featurizer.blocks.12.mlp.fc2.bias": "model-00001-of-00004.safetensors",
368
+ "vision_backbone.featurizer.blocks.12.mlp.fc2.weight": "model-00001-of-00004.safetensors",
369
+ "vision_backbone.featurizer.blocks.12.norm1.bias": "model-00001-of-00004.safetensors",
370
+ "vision_backbone.featurizer.blocks.12.norm1.weight": "model-00001-of-00004.safetensors",
371
+ "vision_backbone.featurizer.blocks.12.norm2.bias": "model-00001-of-00004.safetensors",
372
+ "vision_backbone.featurizer.blocks.12.norm2.weight": "model-00001-of-00004.safetensors",
373
+ "vision_backbone.featurizer.blocks.13.attn.proj.bias": "model-00001-of-00004.safetensors",
374
+ "vision_backbone.featurizer.blocks.13.attn.proj.weight": "model-00001-of-00004.safetensors",
375
+ "vision_backbone.featurizer.blocks.13.attn.qkv.bias": "model-00001-of-00004.safetensors",
376
+ "vision_backbone.featurizer.blocks.13.attn.qkv.weight": "model-00001-of-00004.safetensors",
377
+ "vision_backbone.featurizer.blocks.13.ls1.scale_factor": "model-00001-of-00004.safetensors",
378
+ "vision_backbone.featurizer.blocks.13.ls2.scale_factor": "model-00001-of-00004.safetensors",
379
+ "vision_backbone.featurizer.blocks.13.mlp.fc1.bias": "model-00001-of-00004.safetensors",
380
+ "vision_backbone.featurizer.blocks.13.mlp.fc1.weight": "model-00001-of-00004.safetensors",
381
+ "vision_backbone.featurizer.blocks.13.mlp.fc2.bias": "model-00001-of-00004.safetensors",
382
+ "vision_backbone.featurizer.blocks.13.mlp.fc2.weight": "model-00001-of-00004.safetensors",
383
+ "vision_backbone.featurizer.blocks.13.norm1.bias": "model-00001-of-00004.safetensors",
384
+ "vision_backbone.featurizer.blocks.13.norm1.weight": "model-00001-of-00004.safetensors",
385
+ "vision_backbone.featurizer.blocks.13.norm2.bias": "model-00001-of-00004.safetensors",
386
+ "vision_backbone.featurizer.blocks.13.norm2.weight": "model-00001-of-00004.safetensors",
387
+ "vision_backbone.featurizer.blocks.14.attn.proj.bias": "model-00001-of-00004.safetensors",
388
+ "vision_backbone.featurizer.blocks.14.attn.proj.weight": "model-00001-of-00004.safetensors",
389
+ "vision_backbone.featurizer.blocks.14.attn.qkv.bias": "model-00001-of-00004.safetensors",
390
+ "vision_backbone.featurizer.blocks.14.attn.qkv.weight": "model-00001-of-00004.safetensors",
391
+ "vision_backbone.featurizer.blocks.14.ls1.scale_factor": "model-00001-of-00004.safetensors",
392
+ "vision_backbone.featurizer.blocks.14.ls2.scale_factor": "model-00001-of-00004.safetensors",
393
+ "vision_backbone.featurizer.blocks.14.mlp.fc1.bias": "model-00001-of-00004.safetensors",
394
+ "vision_backbone.featurizer.blocks.14.mlp.fc1.weight": "model-00001-of-00004.safetensors",
395
+ "vision_backbone.featurizer.blocks.14.mlp.fc2.bias": "model-00001-of-00004.safetensors",
396
+ "vision_backbone.featurizer.blocks.14.mlp.fc2.weight": "model-00001-of-00004.safetensors",
397
+ "vision_backbone.featurizer.blocks.14.norm1.bias": "model-00001-of-00004.safetensors",
398
+ "vision_backbone.featurizer.blocks.14.norm1.weight": "model-00001-of-00004.safetensors",
399
+ "vision_backbone.featurizer.blocks.14.norm2.bias": "model-00001-of-00004.safetensors",
400
+ "vision_backbone.featurizer.blocks.14.norm2.weight": "model-00001-of-00004.safetensors",
401
+ "vision_backbone.featurizer.blocks.15.attn.proj.bias": "model-00001-of-00004.safetensors",
402
+ "vision_backbone.featurizer.blocks.15.attn.proj.weight": "model-00001-of-00004.safetensors",
403
+ "vision_backbone.featurizer.blocks.15.attn.qkv.bias": "model-00001-of-00004.safetensors",
404
+ "vision_backbone.featurizer.blocks.15.attn.qkv.weight": "model-00001-of-00004.safetensors",
405
+ "vision_backbone.featurizer.blocks.15.ls1.scale_factor": "model-00001-of-00004.safetensors",
406
+ "vision_backbone.featurizer.blocks.15.ls2.scale_factor": "model-00001-of-00004.safetensors",
407
+ "vision_backbone.featurizer.blocks.15.mlp.fc1.bias": "model-00001-of-00004.safetensors",
408
+ "vision_backbone.featurizer.blocks.15.mlp.fc1.weight": "model-00001-of-00004.safetensors",
409
+ "vision_backbone.featurizer.blocks.15.mlp.fc2.bias": "model-00001-of-00004.safetensors",
410
+ "vision_backbone.featurizer.blocks.15.mlp.fc2.weight": "model-00001-of-00004.safetensors",
411
+ "vision_backbone.featurizer.blocks.15.norm1.bias": "model-00001-of-00004.safetensors",
412
+ "vision_backbone.featurizer.blocks.15.norm1.weight": "model-00001-of-00004.safetensors",
413
+ "vision_backbone.featurizer.blocks.15.norm2.bias": "model-00001-of-00004.safetensors",
414
+ "vision_backbone.featurizer.blocks.15.norm2.weight": "model-00001-of-00004.safetensors",
415
+ "vision_backbone.featurizer.blocks.16.attn.proj.bias": "model-00001-of-00004.safetensors",
416
+ "vision_backbone.featurizer.blocks.16.attn.proj.weight": "model-00001-of-00004.safetensors",
417
+ "vision_backbone.featurizer.blocks.16.attn.qkv.bias": "model-00001-of-00004.safetensors",
418
+ "vision_backbone.featurizer.blocks.16.attn.qkv.weight": "model-00001-of-00004.safetensors",
419
+ "vision_backbone.featurizer.blocks.16.ls1.scale_factor": "model-00001-of-00004.safetensors",
420
+ "vision_backbone.featurizer.blocks.16.ls2.scale_factor": "model-00001-of-00004.safetensors",
421
+ "vision_backbone.featurizer.blocks.16.mlp.fc1.bias": "model-00001-of-00004.safetensors",
422
+ "vision_backbone.featurizer.blocks.16.mlp.fc1.weight": "model-00001-of-00004.safetensors",
423
+ "vision_backbone.featurizer.blocks.16.mlp.fc2.bias": "model-00001-of-00004.safetensors",
424
+ "vision_backbone.featurizer.blocks.16.mlp.fc2.weight": "model-00001-of-00004.safetensors",
425
+ "vision_backbone.featurizer.blocks.16.norm1.bias": "model-00001-of-00004.safetensors",
426
+ "vision_backbone.featurizer.blocks.16.norm1.weight": "model-00001-of-00004.safetensors",
427
+ "vision_backbone.featurizer.blocks.16.norm2.bias": "model-00001-of-00004.safetensors",
428
+ "vision_backbone.featurizer.blocks.16.norm2.weight": "model-00001-of-00004.safetensors",
429
+ "vision_backbone.featurizer.blocks.17.attn.proj.bias": "model-00001-of-00004.safetensors",
430
+ "vision_backbone.featurizer.blocks.17.attn.proj.weight": "model-00001-of-00004.safetensors",
431
+ "vision_backbone.featurizer.blocks.17.attn.qkv.bias": "model-00001-of-00004.safetensors",
432
+ "vision_backbone.featurizer.blocks.17.attn.qkv.weight": "model-00001-of-00004.safetensors",
433
+ "vision_backbone.featurizer.blocks.17.ls1.scale_factor": "model-00001-of-00004.safetensors",
434
+ "vision_backbone.featurizer.blocks.17.ls2.scale_factor": "model-00001-of-00004.safetensors",
435
+ "vision_backbone.featurizer.blocks.17.mlp.fc1.bias": "model-00001-of-00004.safetensors",
436
+ "vision_backbone.featurizer.blocks.17.mlp.fc1.weight": "model-00001-of-00004.safetensors",
437
+ "vision_backbone.featurizer.blocks.17.mlp.fc2.bias": "model-00001-of-00004.safetensors",
438
+ "vision_backbone.featurizer.blocks.17.mlp.fc2.weight": "model-00001-of-00004.safetensors",
439
+ "vision_backbone.featurizer.blocks.17.norm1.bias": "model-00001-of-00004.safetensors",
440
+ "vision_backbone.featurizer.blocks.17.norm1.weight": "model-00001-of-00004.safetensors",
441
+ "vision_backbone.featurizer.blocks.17.norm2.bias": "model-00001-of-00004.safetensors",
442
+ "vision_backbone.featurizer.blocks.17.norm2.weight": "model-00001-of-00004.safetensors",
443
+ "vision_backbone.featurizer.blocks.18.attn.proj.bias": "model-00001-of-00004.safetensors",
444
+ "vision_backbone.featurizer.blocks.18.attn.proj.weight": "model-00001-of-00004.safetensors",
445
+ "vision_backbone.featurizer.blocks.18.attn.qkv.bias": "model-00001-of-00004.safetensors",
446
+ "vision_backbone.featurizer.blocks.18.attn.qkv.weight": "model-00001-of-00004.safetensors",
447
+ "vision_backbone.featurizer.blocks.18.ls1.scale_factor": "model-00001-of-00004.safetensors",
448
+ "vision_backbone.featurizer.blocks.18.ls2.scale_factor": "model-00001-of-00004.safetensors",
449
+ "vision_backbone.featurizer.blocks.18.mlp.fc1.bias": "model-00001-of-00004.safetensors",
450
+ "vision_backbone.featurizer.blocks.18.mlp.fc1.weight": "model-00001-of-00004.safetensors",
451
+ "vision_backbone.featurizer.blocks.18.mlp.fc2.bias": "model-00001-of-00004.safetensors",
452
+ "vision_backbone.featurizer.blocks.18.mlp.fc2.weight": "model-00001-of-00004.safetensors",
453
+ "vision_backbone.featurizer.blocks.18.norm1.bias": "model-00001-of-00004.safetensors",
454
+ "vision_backbone.featurizer.blocks.18.norm1.weight": "model-00001-of-00004.safetensors",
455
+ "vision_backbone.featurizer.blocks.18.norm2.bias": "model-00001-of-00004.safetensors",
456
+ "vision_backbone.featurizer.blocks.18.norm2.weight": "model-00001-of-00004.safetensors",
457
+ "vision_backbone.featurizer.blocks.19.attn.proj.bias": "model-00001-of-00004.safetensors",
458
+ "vision_backbone.featurizer.blocks.19.attn.proj.weight": "model-00001-of-00004.safetensors",
459
+ "vision_backbone.featurizer.blocks.19.attn.qkv.bias": "model-00001-of-00004.safetensors",
460
+ "vision_backbone.featurizer.blocks.19.attn.qkv.weight": "model-00001-of-00004.safetensors",
461
+ "vision_backbone.featurizer.blocks.19.ls1.scale_factor": "model-00001-of-00004.safetensors",
462
+ "vision_backbone.featurizer.blocks.19.ls2.scale_factor": "model-00001-of-00004.safetensors",
463
+ "vision_backbone.featurizer.blocks.19.mlp.fc1.bias": "model-00001-of-00004.safetensors",
464
+ "vision_backbone.featurizer.blocks.19.mlp.fc1.weight": "model-00001-of-00004.safetensors",
465
+ "vision_backbone.featurizer.blocks.19.mlp.fc2.bias": "model-00001-of-00004.safetensors",
466
+ "vision_backbone.featurizer.blocks.19.mlp.fc2.weight": "model-00001-of-00004.safetensors",
467
+ "vision_backbone.featurizer.blocks.19.norm1.bias": "model-00001-of-00004.safetensors",
468
+ "vision_backbone.featurizer.blocks.19.norm1.weight": "model-00001-of-00004.safetensors",
469
+ "vision_backbone.featurizer.blocks.19.norm2.bias": "model-00001-of-00004.safetensors",
470
+ "vision_backbone.featurizer.blocks.19.norm2.weight": "model-00001-of-00004.safetensors",
471
+ "vision_backbone.featurizer.blocks.2.attn.proj.bias": "model-00001-of-00004.safetensors",
472
+ "vision_backbone.featurizer.blocks.2.attn.proj.weight": "model-00001-of-00004.safetensors",
473
+ "vision_backbone.featurizer.blocks.2.attn.qkv.bias": "model-00001-of-00004.safetensors",
474
+ "vision_backbone.featurizer.blocks.2.attn.qkv.weight": "model-00001-of-00004.safetensors",
475
+ "vision_backbone.featurizer.blocks.2.ls1.scale_factor": "model-00001-of-00004.safetensors",
476
+ "vision_backbone.featurizer.blocks.2.ls2.scale_factor": "model-00001-of-00004.safetensors",
477
+ "vision_backbone.featurizer.blocks.2.mlp.fc1.bias": "model-00001-of-00004.safetensors",
478
+ "vision_backbone.featurizer.blocks.2.mlp.fc1.weight": "model-00001-of-00004.safetensors",
479
+ "vision_backbone.featurizer.blocks.2.mlp.fc2.bias": "model-00001-of-00004.safetensors",
480
+ "vision_backbone.featurizer.blocks.2.mlp.fc2.weight": "model-00001-of-00004.safetensors",
481
+ "vision_backbone.featurizer.blocks.2.norm1.bias": "model-00001-of-00004.safetensors",
482
+ "vision_backbone.featurizer.blocks.2.norm1.weight": "model-00001-of-00004.safetensors",
483
+ "vision_backbone.featurizer.blocks.2.norm2.bias": "model-00001-of-00004.safetensors",
484
+ "vision_backbone.featurizer.blocks.2.norm2.weight": "model-00001-of-00004.safetensors",
485
+ "vision_backbone.featurizer.blocks.20.attn.proj.bias": "model-00001-of-00004.safetensors",
486
+ "vision_backbone.featurizer.blocks.20.attn.proj.weight": "model-00001-of-00004.safetensors",
487
+ "vision_backbone.featurizer.blocks.20.attn.qkv.bias": "model-00001-of-00004.safetensors",
488
+ "vision_backbone.featurizer.blocks.20.attn.qkv.weight": "model-00001-of-00004.safetensors",
489
+ "vision_backbone.featurizer.blocks.20.ls1.scale_factor": "model-00001-of-00004.safetensors",
490
+ "vision_backbone.featurizer.blocks.20.ls2.scale_factor": "model-00001-of-00004.safetensors",
491
+ "vision_backbone.featurizer.blocks.20.mlp.fc1.bias": "model-00001-of-00004.safetensors",
492
+ "vision_backbone.featurizer.blocks.20.mlp.fc1.weight": "model-00001-of-00004.safetensors",
493
+ "vision_backbone.featurizer.blocks.20.mlp.fc2.bias": "model-00001-of-00004.safetensors",
494
+ "vision_backbone.featurizer.blocks.20.mlp.fc2.weight": "model-00001-of-00004.safetensors",
495
+ "vision_backbone.featurizer.blocks.20.norm1.bias": "model-00001-of-00004.safetensors",
496
+ "vision_backbone.featurizer.blocks.20.norm1.weight": "model-00001-of-00004.safetensors",
497
+ "vision_backbone.featurizer.blocks.20.norm2.bias": "model-00001-of-00004.safetensors",
498
+ "vision_backbone.featurizer.blocks.20.norm2.weight": "model-00001-of-00004.safetensors",
499
+ "vision_backbone.featurizer.blocks.21.attn.proj.bias": "model-00001-of-00004.safetensors",
500
+ "vision_backbone.featurizer.blocks.21.attn.proj.weight": "model-00001-of-00004.safetensors",
501
+ "vision_backbone.featurizer.blocks.21.attn.qkv.bias": "model-00001-of-00004.safetensors",
502
+ "vision_backbone.featurizer.blocks.21.attn.qkv.weight": "model-00001-of-00004.safetensors",
503
+ "vision_backbone.featurizer.blocks.21.ls1.scale_factor": "model-00001-of-00004.safetensors",
504
+ "vision_backbone.featurizer.blocks.21.ls2.scale_factor": "model-00001-of-00004.safetensors",
505
+ "vision_backbone.featurizer.blocks.21.mlp.fc1.bias": "model-00001-of-00004.safetensors",
506
+ "vision_backbone.featurizer.blocks.21.mlp.fc1.weight": "model-00001-of-00004.safetensors",
507
+ "vision_backbone.featurizer.blocks.21.mlp.fc2.bias": "model-00001-of-00004.safetensors",
508
+ "vision_backbone.featurizer.blocks.21.mlp.fc2.weight": "model-00001-of-00004.safetensors",
509
+ "vision_backbone.featurizer.blocks.21.norm1.bias": "model-00001-of-00004.safetensors",
510
+ "vision_backbone.featurizer.blocks.21.norm1.weight": "model-00001-of-00004.safetensors",
511
+ "vision_backbone.featurizer.blocks.21.norm2.bias": "model-00001-of-00004.safetensors",
512
+ "vision_backbone.featurizer.blocks.21.norm2.weight": "model-00001-of-00004.safetensors",
513
+ "vision_backbone.featurizer.blocks.22.attn.proj.bias": "model-00001-of-00004.safetensors",
514
+ "vision_backbone.featurizer.blocks.22.attn.proj.weight": "model-00001-of-00004.safetensors",
515
+ "vision_backbone.featurizer.blocks.22.attn.qkv.bias": "model-00001-of-00004.safetensors",
516
+ "vision_backbone.featurizer.blocks.22.attn.qkv.weight": "model-00001-of-00004.safetensors",
517
+ "vision_backbone.featurizer.blocks.22.ls1.scale_factor": "model-00001-of-00004.safetensors",
518
+ "vision_backbone.featurizer.blocks.22.ls2.scale_factor": "model-00001-of-00004.safetensors",
519
+ "vision_backbone.featurizer.blocks.22.mlp.fc1.bias": "model-00001-of-00004.safetensors",
520
+ "vision_backbone.featurizer.blocks.22.mlp.fc1.weight": "model-00001-of-00004.safetensors",
521
+ "vision_backbone.featurizer.blocks.22.mlp.fc2.bias": "model-00001-of-00004.safetensors",
522
+ "vision_backbone.featurizer.blocks.22.mlp.fc2.weight": "model-00001-of-00004.safetensors",
523
+ "vision_backbone.featurizer.blocks.22.norm1.bias": "model-00001-of-00004.safetensors",
524
+ "vision_backbone.featurizer.blocks.22.norm1.weight": "model-00001-of-00004.safetensors",
525
+ "vision_backbone.featurizer.blocks.22.norm2.bias": "model-00001-of-00004.safetensors",
526
+ "vision_backbone.featurizer.blocks.22.norm2.weight": "model-00001-of-00004.safetensors",
527
+ "vision_backbone.featurizer.blocks.23.attn.proj.bias": "model-00001-of-00004.safetensors",
528
+ "vision_backbone.featurizer.blocks.23.attn.proj.weight": "model-00001-of-00004.safetensors",
529
+ "vision_backbone.featurizer.blocks.23.attn.qkv.bias": "model-00001-of-00004.safetensors",
530
+ "vision_backbone.featurizer.blocks.23.attn.qkv.weight": "model-00001-of-00004.safetensors",
531
+ "vision_backbone.featurizer.blocks.23.ls1.scale_factor": "model-00001-of-00004.safetensors",
532
+ "vision_backbone.featurizer.blocks.23.ls2.scale_factor": "model-00001-of-00004.safetensors",
533
+ "vision_backbone.featurizer.blocks.23.mlp.fc1.bias": "model-00001-of-00004.safetensors",
534
+ "vision_backbone.featurizer.blocks.23.mlp.fc1.weight": "model-00001-of-00004.safetensors",
535
+ "vision_backbone.featurizer.blocks.23.mlp.fc2.bias": "model-00001-of-00004.safetensors",
536
+ "vision_backbone.featurizer.blocks.23.mlp.fc2.weight": "model-00001-of-00004.safetensors",
537
+ "vision_backbone.featurizer.blocks.23.norm1.bias": "model-00001-of-00004.safetensors",
538
+ "vision_backbone.featurizer.blocks.23.norm1.weight": "model-00001-of-00004.safetensors",
539
+ "vision_backbone.featurizer.blocks.23.norm2.bias": "model-00001-of-00004.safetensors",
540
+ "vision_backbone.featurizer.blocks.23.norm2.weight": "model-00001-of-00004.safetensors",
541
+ "vision_backbone.featurizer.blocks.3.attn.proj.bias": "model-00001-of-00004.safetensors",
542
+ "vision_backbone.featurizer.blocks.3.attn.proj.weight": "model-00001-of-00004.safetensors",
543
+ "vision_backbone.featurizer.blocks.3.attn.qkv.bias": "model-00001-of-00004.safetensors",
544
+ "vision_backbone.featurizer.blocks.3.attn.qkv.weight": "model-00001-of-00004.safetensors",
545
+ "vision_backbone.featurizer.blocks.3.ls1.scale_factor": "model-00001-of-00004.safetensors",
546
+ "vision_backbone.featurizer.blocks.3.ls2.scale_factor": "model-00001-of-00004.safetensors",
547
+ "vision_backbone.featurizer.blocks.3.mlp.fc1.bias": "model-00001-of-00004.safetensors",
548
+ "vision_backbone.featurizer.blocks.3.mlp.fc1.weight": "model-00001-of-00004.safetensors",
549
+ "vision_backbone.featurizer.blocks.3.mlp.fc2.bias": "model-00001-of-00004.safetensors",
550
+ "vision_backbone.featurizer.blocks.3.mlp.fc2.weight": "model-00001-of-00004.safetensors",
551
+ "vision_backbone.featurizer.blocks.3.norm1.bias": "model-00001-of-00004.safetensors",
552
+ "vision_backbone.featurizer.blocks.3.norm1.weight": "model-00001-of-00004.safetensors",
553
+ "vision_backbone.featurizer.blocks.3.norm2.bias": "model-00001-of-00004.safetensors",
554
+ "vision_backbone.featurizer.blocks.3.norm2.weight": "model-00001-of-00004.safetensors",
555
+ "vision_backbone.featurizer.blocks.4.attn.proj.bias": "model-00001-of-00004.safetensors",
556
+ "vision_backbone.featurizer.blocks.4.attn.proj.weight": "model-00001-of-00004.safetensors",
557
+ "vision_backbone.featurizer.blocks.4.attn.qkv.bias": "model-00001-of-00004.safetensors",
558
+ "vision_backbone.featurizer.blocks.4.attn.qkv.weight": "model-00001-of-00004.safetensors",
559
+ "vision_backbone.featurizer.blocks.4.ls1.scale_factor": "model-00001-of-00004.safetensors",
560
+ "vision_backbone.featurizer.blocks.4.ls2.scale_factor": "model-00001-of-00004.safetensors",
561
+ "vision_backbone.featurizer.blocks.4.mlp.fc1.bias": "model-00001-of-00004.safetensors",
562
+ "vision_backbone.featurizer.blocks.4.mlp.fc1.weight": "model-00001-of-00004.safetensors",
563
+ "vision_backbone.featurizer.blocks.4.mlp.fc2.bias": "model-00001-of-00004.safetensors",
564
+ "vision_backbone.featurizer.blocks.4.mlp.fc2.weight": "model-00001-of-00004.safetensors",
565
+ "vision_backbone.featurizer.blocks.4.norm1.bias": "model-00001-of-00004.safetensors",
566
+ "vision_backbone.featurizer.blocks.4.norm1.weight": "model-00001-of-00004.safetensors",
567
+ "vision_backbone.featurizer.blocks.4.norm2.bias": "model-00001-of-00004.safetensors",
568
+ "vision_backbone.featurizer.blocks.4.norm2.weight": "model-00001-of-00004.safetensors",
569
+ "vision_backbone.featurizer.blocks.5.attn.proj.bias": "model-00001-of-00004.safetensors",
570
+ "vision_backbone.featurizer.blocks.5.attn.proj.weight": "model-00001-of-00004.safetensors",
571
+ "vision_backbone.featurizer.blocks.5.attn.qkv.bias": "model-00001-of-00004.safetensors",
572
+ "vision_backbone.featurizer.blocks.5.attn.qkv.weight": "model-00001-of-00004.safetensors",
573
+ "vision_backbone.featurizer.blocks.5.ls1.scale_factor": "model-00001-of-00004.safetensors",
574
+ "vision_backbone.featurizer.blocks.5.ls2.scale_factor": "model-00001-of-00004.safetensors",
575
+ "vision_backbone.featurizer.blocks.5.mlp.fc1.bias": "model-00001-of-00004.safetensors",
576
+ "vision_backbone.featurizer.blocks.5.mlp.fc1.weight": "model-00001-of-00004.safetensors",
577
+ "vision_backbone.featurizer.blocks.5.mlp.fc2.bias": "model-00001-of-00004.safetensors",
578
+ "vision_backbone.featurizer.blocks.5.mlp.fc2.weight": "model-00001-of-00004.safetensors",
579
+ "vision_backbone.featurizer.blocks.5.norm1.bias": "model-00001-of-00004.safetensors",
580
+ "vision_backbone.featurizer.blocks.5.norm1.weight": "model-00001-of-00004.safetensors",
581
+ "vision_backbone.featurizer.blocks.5.norm2.bias": "model-00001-of-00004.safetensors",
582
+ "vision_backbone.featurizer.blocks.5.norm2.weight": "model-00001-of-00004.safetensors",
583
+ "vision_backbone.featurizer.blocks.6.attn.proj.bias": "model-00001-of-00004.safetensors",
584
+ "vision_backbone.featurizer.blocks.6.attn.proj.weight": "model-00001-of-00004.safetensors",
585
+ "vision_backbone.featurizer.blocks.6.attn.qkv.bias": "model-00001-of-00004.safetensors",
586
+ "vision_backbone.featurizer.blocks.6.attn.qkv.weight": "model-00001-of-00004.safetensors",
587
+ "vision_backbone.featurizer.blocks.6.ls1.scale_factor": "model-00001-of-00004.safetensors",
588
+ "vision_backbone.featurizer.blocks.6.ls2.scale_factor": "model-00001-of-00004.safetensors",
589
+ "vision_backbone.featurizer.blocks.6.mlp.fc1.bias": "model-00001-of-00004.safetensors",
590
+ "vision_backbone.featurizer.blocks.6.mlp.fc1.weight": "model-00001-of-00004.safetensors",
591
+ "vision_backbone.featurizer.blocks.6.mlp.fc2.bias": "model-00001-of-00004.safetensors",
592
+ "vision_backbone.featurizer.blocks.6.mlp.fc2.weight": "model-00001-of-00004.safetensors",
593
+ "vision_backbone.featurizer.blocks.6.norm1.bias": "model-00001-of-00004.safetensors",
594
+ "vision_backbone.featurizer.blocks.6.norm1.weight": "model-00001-of-00004.safetensors",
595
+ "vision_backbone.featurizer.blocks.6.norm2.bias": "model-00001-of-00004.safetensors",
596
+ "vision_backbone.featurizer.blocks.6.norm2.weight": "model-00001-of-00004.safetensors",
597
+ "vision_backbone.featurizer.blocks.7.attn.proj.bias": "model-00001-of-00004.safetensors",
598
+ "vision_backbone.featurizer.blocks.7.attn.proj.weight": "model-00001-of-00004.safetensors",
599
+ "vision_backbone.featurizer.blocks.7.attn.qkv.bias": "model-00001-of-00004.safetensors",
600
+ "vision_backbone.featurizer.blocks.7.attn.qkv.weight": "model-00001-of-00004.safetensors",
601
+ "vision_backbone.featurizer.blocks.7.ls1.scale_factor": "model-00001-of-00004.safetensors",
602
+ "vision_backbone.featurizer.blocks.7.ls2.scale_factor": "model-00001-of-00004.safetensors",
603
+ "vision_backbone.featurizer.blocks.7.mlp.fc1.bias": "model-00001-of-00004.safetensors",
604
+ "vision_backbone.featurizer.blocks.7.mlp.fc1.weight": "model-00001-of-00004.safetensors",
605
+ "vision_backbone.featurizer.blocks.7.mlp.fc2.bias": "model-00001-of-00004.safetensors",
606
+ "vision_backbone.featurizer.blocks.7.mlp.fc2.weight": "model-00001-of-00004.safetensors",
607
+ "vision_backbone.featurizer.blocks.7.norm1.bias": "model-00001-of-00004.safetensors",
608
+ "vision_backbone.featurizer.blocks.7.norm1.weight": "model-00001-of-00004.safetensors",
609
+ "vision_backbone.featurizer.blocks.7.norm2.bias": "model-00001-of-00004.safetensors",
610
+ "vision_backbone.featurizer.blocks.7.norm2.weight": "model-00001-of-00004.safetensors",
611
+ "vision_backbone.featurizer.blocks.8.attn.proj.bias": "model-00001-of-00004.safetensors",
612
+ "vision_backbone.featurizer.blocks.8.attn.proj.weight": "model-00001-of-00004.safetensors",
613
+ "vision_backbone.featurizer.blocks.8.attn.qkv.bias": "model-00001-of-00004.safetensors",
614
+ "vision_backbone.featurizer.blocks.8.attn.qkv.weight": "model-00001-of-00004.safetensors",
615
+ "vision_backbone.featurizer.blocks.8.ls1.scale_factor": "model-00001-of-00004.safetensors",
616
+ "vision_backbone.featurizer.blocks.8.ls2.scale_factor": "model-00001-of-00004.safetensors",
617
+ "vision_backbone.featurizer.blocks.8.mlp.fc1.bias": "model-00001-of-00004.safetensors",
618
+ "vision_backbone.featurizer.blocks.8.mlp.fc1.weight": "model-00001-of-00004.safetensors",
619
+ "vision_backbone.featurizer.blocks.8.mlp.fc2.bias": "model-00001-of-00004.safetensors",
620
+ "vision_backbone.featurizer.blocks.8.mlp.fc2.weight": "model-00001-of-00004.safetensors",
621
+ "vision_backbone.featurizer.blocks.8.norm1.bias": "model-00001-of-00004.safetensors",
622
+ "vision_backbone.featurizer.blocks.8.norm1.weight": "model-00001-of-00004.safetensors",
623
+ "vision_backbone.featurizer.blocks.8.norm2.bias": "model-00001-of-00004.safetensors",
624
+ "vision_backbone.featurizer.blocks.8.norm2.weight": "model-00001-of-00004.safetensors",
625
+ "vision_backbone.featurizer.blocks.9.attn.proj.bias": "model-00001-of-00004.safetensors",
626
+ "vision_backbone.featurizer.blocks.9.attn.proj.weight": "model-00001-of-00004.safetensors",
627
+ "vision_backbone.featurizer.blocks.9.attn.qkv.bias": "model-00001-of-00004.safetensors",
628
+ "vision_backbone.featurizer.blocks.9.attn.qkv.weight": "model-00001-of-00004.safetensors",
629
+ "vision_backbone.featurizer.blocks.9.ls1.scale_factor": "model-00001-of-00004.safetensors",
630
+ "vision_backbone.featurizer.blocks.9.ls2.scale_factor": "model-00001-of-00004.safetensors",
631
+ "vision_backbone.featurizer.blocks.9.mlp.fc1.bias": "model-00001-of-00004.safetensors",
632
+ "vision_backbone.featurizer.blocks.9.mlp.fc1.weight": "model-00001-of-00004.safetensors",
633
+ "vision_backbone.featurizer.blocks.9.mlp.fc2.bias": "model-00001-of-00004.safetensors",
634
+ "vision_backbone.featurizer.blocks.9.mlp.fc2.weight": "model-00001-of-00004.safetensors",
635
+ "vision_backbone.featurizer.blocks.9.norm1.bias": "model-00001-of-00004.safetensors",
636
+ "vision_backbone.featurizer.blocks.9.norm1.weight": "model-00001-of-00004.safetensors",
637
+ "vision_backbone.featurizer.blocks.9.norm2.bias": "model-00001-of-00004.safetensors",
638
+ "vision_backbone.featurizer.blocks.9.norm2.weight": "model-00001-of-00004.safetensors",
639
+ "vision_backbone.featurizer.cls_token": "model-00001-of-00004.safetensors",
640
+ "vision_backbone.featurizer.norm.bias": "model-00001-of-00004.safetensors",
641
+ "vision_backbone.featurizer.norm.weight": "model-00001-of-00004.safetensors",
642
+ "vision_backbone.featurizer.patch_embed.proj.bias": "model-00001-of-00004.safetensors",
643
+ "vision_backbone.featurizer.patch_embed.proj.weight": "model-00001-of-00004.safetensors",
644
+ "vision_backbone.featurizer.pos_embed": "model-00001-of-00004.safetensors",
645
+ "vision_backbone.featurizer.reg_token": "model-00001-of-00004.safetensors",
646
+ "vision_backbone.fused_featurizer.attn_pool.kv.bias": "model-00001-of-00004.safetensors",
647
+ "vision_backbone.fused_featurizer.attn_pool.kv.weight": "model-00001-of-00004.safetensors",
648
+ "vision_backbone.fused_featurizer.attn_pool.latent": "model-00001-of-00004.safetensors",
649
+ "vision_backbone.fused_featurizer.attn_pool.mlp.fc1.bias": "model-00001-of-00004.safetensors",
650
+ "vision_backbone.fused_featurizer.attn_pool.mlp.fc1.weight": "model-00001-of-00004.safetensors",
651
+ "vision_backbone.fused_featurizer.attn_pool.mlp.fc2.bias": "model-00001-of-00004.safetensors",
652
+ "vision_backbone.fused_featurizer.attn_pool.mlp.fc2.weight": "model-00001-of-00004.safetensors",
653
+ "vision_backbone.fused_featurizer.attn_pool.norm.bias": "model-00001-of-00004.safetensors",
654
+ "vision_backbone.fused_featurizer.attn_pool.norm.weight": "model-00001-of-00004.safetensors",
655
+ "vision_backbone.fused_featurizer.attn_pool.proj.bias": "model-00001-of-00004.safetensors",
656
+ "vision_backbone.fused_featurizer.attn_pool.proj.weight": "model-00001-of-00004.safetensors",
657
+ "vision_backbone.fused_featurizer.attn_pool.q.bias": "model-00001-of-00004.safetensors",
658
+ "vision_backbone.fused_featurizer.attn_pool.q.weight": "model-00001-of-00004.safetensors",
659
+ "vision_backbone.fused_featurizer.blocks.0.attn.proj.bias": "model-00001-of-00004.safetensors",
660
+ "vision_backbone.fused_featurizer.blocks.0.attn.proj.weight": "model-00001-of-00004.safetensors",
661
+ "vision_backbone.fused_featurizer.blocks.0.attn.qkv.bias": "model-00001-of-00004.safetensors",
662
+ "vision_backbone.fused_featurizer.blocks.0.attn.qkv.weight": "model-00001-of-00004.safetensors",
663
+ "vision_backbone.fused_featurizer.blocks.0.mlp.fc1.bias": "model-00001-of-00004.safetensors",
664
+ "vision_backbone.fused_featurizer.blocks.0.mlp.fc1.weight": "model-00001-of-00004.safetensors",
665
+ "vision_backbone.fused_featurizer.blocks.0.mlp.fc2.bias": "model-00001-of-00004.safetensors",
666
+ "vision_backbone.fused_featurizer.blocks.0.mlp.fc2.weight": "model-00001-of-00004.safetensors",
667
+ "vision_backbone.fused_featurizer.blocks.0.norm1.bias": "model-00001-of-00004.safetensors",
668
+ "vision_backbone.fused_featurizer.blocks.0.norm1.weight": "model-00001-of-00004.safetensors",
669
+ "vision_backbone.fused_featurizer.blocks.0.norm2.bias": "model-00001-of-00004.safetensors",
670
+ "vision_backbone.fused_featurizer.blocks.0.norm2.weight": "model-00001-of-00004.safetensors",
671
+ "vision_backbone.fused_featurizer.blocks.1.attn.proj.bias": "model-00001-of-00004.safetensors",
672
+ "vision_backbone.fused_featurizer.blocks.1.attn.proj.weight": "model-00001-of-00004.safetensors",
673
+ "vision_backbone.fused_featurizer.blocks.1.attn.qkv.bias": "model-00001-of-00004.safetensors",
674
+ "vision_backbone.fused_featurizer.blocks.1.attn.qkv.weight": "model-00001-of-00004.safetensors",
675
+ "vision_backbone.fused_featurizer.blocks.1.mlp.fc1.bias": "model-00001-of-00004.safetensors",
676
+ "vision_backbone.fused_featurizer.blocks.1.mlp.fc1.weight": "model-00001-of-00004.safetensors",
677
+ "vision_backbone.fused_featurizer.blocks.1.mlp.fc2.bias": "model-00001-of-00004.safetensors",
678
+ "vision_backbone.fused_featurizer.blocks.1.mlp.fc2.weight": "model-00001-of-00004.safetensors",
679
+ "vision_backbone.fused_featurizer.blocks.1.norm1.bias": "model-00001-of-00004.safetensors",
680
+ "vision_backbone.fused_featurizer.blocks.1.norm1.weight": "model-00001-of-00004.safetensors",
681
+ "vision_backbone.fused_featurizer.blocks.1.norm2.bias": "model-00001-of-00004.safetensors",
682
+ "vision_backbone.fused_featurizer.blocks.1.norm2.weight": "model-00001-of-00004.safetensors",
683
+ "vision_backbone.fused_featurizer.blocks.10.attn.proj.bias": "model-00001-of-00004.safetensors",
684
+ "vision_backbone.fused_featurizer.blocks.10.attn.proj.weight": "model-00001-of-00004.safetensors",
685
+ "vision_backbone.fused_featurizer.blocks.10.attn.qkv.bias": "model-00001-of-00004.safetensors",
686
+ "vision_backbone.fused_featurizer.blocks.10.attn.qkv.weight": "model-00001-of-00004.safetensors",
687
+ "vision_backbone.fused_featurizer.blocks.10.mlp.fc1.bias": "model-00001-of-00004.safetensors",
688
+ "vision_backbone.fused_featurizer.blocks.10.mlp.fc1.weight": "model-00001-of-00004.safetensors",
689
+ "vision_backbone.fused_featurizer.blocks.10.mlp.fc2.bias": "model-00001-of-00004.safetensors",
690
+ "vision_backbone.fused_featurizer.blocks.10.mlp.fc2.weight": "model-00001-of-00004.safetensors",
691
+ "vision_backbone.fused_featurizer.blocks.10.norm1.bias": "model-00001-of-00004.safetensors",
692
+ "vision_backbone.fused_featurizer.blocks.10.norm1.weight": "model-00001-of-00004.safetensors",
693
+ "vision_backbone.fused_featurizer.blocks.10.norm2.bias": "model-00001-of-00004.safetensors",
694
+ "vision_backbone.fused_featurizer.blocks.10.norm2.weight": "model-00001-of-00004.safetensors",
695
+ "vision_backbone.fused_featurizer.blocks.11.attn.proj.bias": "model-00001-of-00004.safetensors",
696
+ "vision_backbone.fused_featurizer.blocks.11.attn.proj.weight": "model-00001-of-00004.safetensors",
697
+ "vision_backbone.fused_featurizer.blocks.11.attn.qkv.bias": "model-00001-of-00004.safetensors",
698
+ "vision_backbone.fused_featurizer.blocks.11.attn.qkv.weight": "model-00001-of-00004.safetensors",
699
+ "vision_backbone.fused_featurizer.blocks.11.mlp.fc1.bias": "model-00001-of-00004.safetensors",
700
+ "vision_backbone.fused_featurizer.blocks.11.mlp.fc1.weight": "model-00001-of-00004.safetensors",
701
+ "vision_backbone.fused_featurizer.blocks.11.mlp.fc2.bias": "model-00001-of-00004.safetensors",
702
+ "vision_backbone.fused_featurizer.blocks.11.mlp.fc2.weight": "model-00001-of-00004.safetensors",
703
+ "vision_backbone.fused_featurizer.blocks.11.norm1.bias": "model-00001-of-00004.safetensors",
704
+ "vision_backbone.fused_featurizer.blocks.11.norm1.weight": "model-00001-of-00004.safetensors",
705
+ "vision_backbone.fused_featurizer.blocks.11.norm2.bias": "model-00001-of-00004.safetensors",
706
+ "vision_backbone.fused_featurizer.blocks.11.norm2.weight": "model-00001-of-00004.safetensors",
707
+ "vision_backbone.fused_featurizer.blocks.12.attn.proj.bias": "model-00001-of-00004.safetensors",
708
+ "vision_backbone.fused_featurizer.blocks.12.attn.proj.weight": "model-00001-of-00004.safetensors",
709
+ "vision_backbone.fused_featurizer.blocks.12.attn.qkv.bias": "model-00001-of-00004.safetensors",
710
+ "vision_backbone.fused_featurizer.blocks.12.attn.qkv.weight": "model-00001-of-00004.safetensors",
711
+ "vision_backbone.fused_featurizer.blocks.12.mlp.fc1.bias": "model-00001-of-00004.safetensors",
712
+ "vision_backbone.fused_featurizer.blocks.12.mlp.fc1.weight": "model-00001-of-00004.safetensors",
713
+ "vision_backbone.fused_featurizer.blocks.12.mlp.fc2.bias": "model-00001-of-00004.safetensors",
714
+ "vision_backbone.fused_featurizer.blocks.12.mlp.fc2.weight": "model-00001-of-00004.safetensors",
715
+ "vision_backbone.fused_featurizer.blocks.12.norm1.bias": "model-00001-of-00004.safetensors",
716
+ "vision_backbone.fused_featurizer.blocks.12.norm1.weight": "model-00001-of-00004.safetensors",
717
+ "vision_backbone.fused_featurizer.blocks.12.norm2.bias": "model-00001-of-00004.safetensors",
718
+ "vision_backbone.fused_featurizer.blocks.12.norm2.weight": "model-00001-of-00004.safetensors",
719
+ "vision_backbone.fused_featurizer.blocks.13.attn.proj.bias": "model-00001-of-00004.safetensors",
720
+ "vision_backbone.fused_featurizer.blocks.13.attn.proj.weight": "model-00001-of-00004.safetensors",
721
+ "vision_backbone.fused_featurizer.blocks.13.attn.qkv.bias": "model-00001-of-00004.safetensors",
722
+ "vision_backbone.fused_featurizer.blocks.13.attn.qkv.weight": "model-00001-of-00004.safetensors",
723
+ "vision_backbone.fused_featurizer.blocks.13.mlp.fc1.bias": "model-00001-of-00004.safetensors",
724
+ "vision_backbone.fused_featurizer.blocks.13.mlp.fc1.weight": "model-00001-of-00004.safetensors",
725
+ "vision_backbone.fused_featurizer.blocks.13.mlp.fc2.bias": "model-00001-of-00004.safetensors",
726
+ "vision_backbone.fused_featurizer.blocks.13.mlp.fc2.weight": "model-00001-of-00004.safetensors",
727
+ "vision_backbone.fused_featurizer.blocks.13.norm1.bias": "model-00001-of-00004.safetensors",
728
+ "vision_backbone.fused_featurizer.blocks.13.norm1.weight": "model-00001-of-00004.safetensors",
729
+ "vision_backbone.fused_featurizer.blocks.13.norm2.bias": "model-00001-of-00004.safetensors",
730
+ "vision_backbone.fused_featurizer.blocks.13.norm2.weight": "model-00001-of-00004.safetensors",
731
+ "vision_backbone.fused_featurizer.blocks.14.attn.proj.bias": "model-00001-of-00004.safetensors",
732
+ "vision_backbone.fused_featurizer.blocks.14.attn.proj.weight": "model-00001-of-00004.safetensors",
733
+ "vision_backbone.fused_featurizer.blocks.14.attn.qkv.bias": "model-00001-of-00004.safetensors",
734
+ "vision_backbone.fused_featurizer.blocks.14.attn.qkv.weight": "model-00001-of-00004.safetensors",
735
+ "vision_backbone.fused_featurizer.blocks.14.mlp.fc1.bias": "model-00001-of-00004.safetensors",
736
+ "vision_backbone.fused_featurizer.blocks.14.mlp.fc1.weight": "model-00001-of-00004.safetensors",
737
+ "vision_backbone.fused_featurizer.blocks.14.mlp.fc2.bias": "model-00001-of-00004.safetensors",
738
+ "vision_backbone.fused_featurizer.blocks.14.mlp.fc2.weight": "model-00001-of-00004.safetensors",
739
+ "vision_backbone.fused_featurizer.blocks.14.norm1.bias": "model-00001-of-00004.safetensors",
740
+ "vision_backbone.fused_featurizer.blocks.14.norm1.weight": "model-00001-of-00004.safetensors",
741
+ "vision_backbone.fused_featurizer.blocks.14.norm2.bias": "model-00001-of-00004.safetensors",
742
+ "vision_backbone.fused_featurizer.blocks.14.norm2.weight": "model-00001-of-00004.safetensors",
743
+ "vision_backbone.fused_featurizer.blocks.15.attn.proj.bias": "model-00001-of-00004.safetensors",
744
+ "vision_backbone.fused_featurizer.blocks.15.attn.proj.weight": "model-00001-of-00004.safetensors",
745
+ "vision_backbone.fused_featurizer.blocks.15.attn.qkv.bias": "model-00001-of-00004.safetensors",
746
+ "vision_backbone.fused_featurizer.blocks.15.attn.qkv.weight": "model-00001-of-00004.safetensors",
747
+ "vision_backbone.fused_featurizer.blocks.15.mlp.fc1.bias": "model-00001-of-00004.safetensors",
748
+ "vision_backbone.fused_featurizer.blocks.15.mlp.fc1.weight": "model-00001-of-00004.safetensors",
749
+ "vision_backbone.fused_featurizer.blocks.15.mlp.fc2.bias": "model-00001-of-00004.safetensors",
750
+ "vision_backbone.fused_featurizer.blocks.15.mlp.fc2.weight": "model-00001-of-00004.safetensors",
751
+ "vision_backbone.fused_featurizer.blocks.15.norm1.bias": "model-00001-of-00004.safetensors",
752
+ "vision_backbone.fused_featurizer.blocks.15.norm1.weight": "model-00001-of-00004.safetensors",
753
+ "vision_backbone.fused_featurizer.blocks.15.norm2.bias": "model-00001-of-00004.safetensors",
754
+ "vision_backbone.fused_featurizer.blocks.15.norm2.weight": "model-00001-of-00004.safetensors",
755
+ "vision_backbone.fused_featurizer.blocks.16.attn.proj.bias": "model-00001-of-00004.safetensors",
756
+ "vision_backbone.fused_featurizer.blocks.16.attn.proj.weight": "model-00001-of-00004.safetensors",
757
+ "vision_backbone.fused_featurizer.blocks.16.attn.qkv.bias": "model-00001-of-00004.safetensors",
758
+ "vision_backbone.fused_featurizer.blocks.16.attn.qkv.weight": "model-00001-of-00004.safetensors",
759
+ "vision_backbone.fused_featurizer.blocks.16.mlp.fc1.bias": "model-00001-of-00004.safetensors",
760
+ "vision_backbone.fused_featurizer.blocks.16.mlp.fc1.weight": "model-00001-of-00004.safetensors",
761
+ "vision_backbone.fused_featurizer.blocks.16.mlp.fc2.bias": "model-00001-of-00004.safetensors",
762
+ "vision_backbone.fused_featurizer.blocks.16.mlp.fc2.weight": "model-00001-of-00004.safetensors",
763
+ "vision_backbone.fused_featurizer.blocks.16.norm1.bias": "model-00001-of-00004.safetensors",
764
+ "vision_backbone.fused_featurizer.blocks.16.norm1.weight": "model-00001-of-00004.safetensors",
765
+ "vision_backbone.fused_featurizer.blocks.16.norm2.bias": "model-00001-of-00004.safetensors",
766
+ "vision_backbone.fused_featurizer.blocks.16.norm2.weight": "model-00001-of-00004.safetensors",
767
+ "vision_backbone.fused_featurizer.blocks.17.attn.proj.bias": "model-00001-of-00004.safetensors",
768
+ "vision_backbone.fused_featurizer.blocks.17.attn.proj.weight": "model-00001-of-00004.safetensors",
769
+ "vision_backbone.fused_featurizer.blocks.17.attn.qkv.bias": "model-00001-of-00004.safetensors",
770
+ "vision_backbone.fused_featurizer.blocks.17.attn.qkv.weight": "model-00001-of-00004.safetensors",
771
+ "vision_backbone.fused_featurizer.blocks.17.mlp.fc1.bias": "model-00001-of-00004.safetensors",
772
+ "vision_backbone.fused_featurizer.blocks.17.mlp.fc1.weight": "model-00001-of-00004.safetensors",
773
+ "vision_backbone.fused_featurizer.blocks.17.mlp.fc2.bias": "model-00001-of-00004.safetensors",
774
+ "vision_backbone.fused_featurizer.blocks.17.mlp.fc2.weight": "model-00001-of-00004.safetensors",
775
+ "vision_backbone.fused_featurizer.blocks.17.norm1.bias": "model-00001-of-00004.safetensors",
776
+ "vision_backbone.fused_featurizer.blocks.17.norm1.weight": "model-00001-of-00004.safetensors",
777
+ "vision_backbone.fused_featurizer.blocks.17.norm2.bias": "model-00001-of-00004.safetensors",
778
+ "vision_backbone.fused_featurizer.blocks.17.norm2.weight": "model-00001-of-00004.safetensors",
779
+ "vision_backbone.fused_featurizer.blocks.18.attn.proj.bias": "model-00001-of-00004.safetensors",
780
+ "vision_backbone.fused_featurizer.blocks.18.attn.proj.weight": "model-00001-of-00004.safetensors",
781
+ "vision_backbone.fused_featurizer.blocks.18.attn.qkv.bias": "model-00001-of-00004.safetensors",
782
+ "vision_backbone.fused_featurizer.blocks.18.attn.qkv.weight": "model-00001-of-00004.safetensors",
783
+ "vision_backbone.fused_featurizer.blocks.18.mlp.fc1.bias": "model-00001-of-00004.safetensors",
784
+ "vision_backbone.fused_featurizer.blocks.18.mlp.fc1.weight": "model-00001-of-00004.safetensors",
785
+ "vision_backbone.fused_featurizer.blocks.18.mlp.fc2.bias": "model-00001-of-00004.safetensors",
786
+ "vision_backbone.fused_featurizer.blocks.18.mlp.fc2.weight": "model-00001-of-00004.safetensors",
787
+ "vision_backbone.fused_featurizer.blocks.18.norm1.bias": "model-00001-of-00004.safetensors",
788
+ "vision_backbone.fused_featurizer.blocks.18.norm1.weight": "model-00001-of-00004.safetensors",
789
+ "vision_backbone.fused_featurizer.blocks.18.norm2.bias": "model-00001-of-00004.safetensors",
790
+ "vision_backbone.fused_featurizer.blocks.18.norm2.weight": "model-00001-of-00004.safetensors",
791
+ "vision_backbone.fused_featurizer.blocks.19.attn.proj.bias": "model-00001-of-00004.safetensors",
792
+ "vision_backbone.fused_featurizer.blocks.19.attn.proj.weight": "model-00001-of-00004.safetensors",
793
+ "vision_backbone.fused_featurizer.blocks.19.attn.qkv.bias": "model-00001-of-00004.safetensors",
794
+ "vision_backbone.fused_featurizer.blocks.19.attn.qkv.weight": "model-00001-of-00004.safetensors",
795
+ "vision_backbone.fused_featurizer.blocks.19.mlp.fc1.bias": "model-00001-of-00004.safetensors",
796
+ "vision_backbone.fused_featurizer.blocks.19.mlp.fc1.weight": "model-00001-of-00004.safetensors",
797
+ "vision_backbone.fused_featurizer.blocks.19.mlp.fc2.bias": "model-00001-of-00004.safetensors",
798
+ "vision_backbone.fused_featurizer.blocks.19.mlp.fc2.weight": "model-00001-of-00004.safetensors",
799
+ "vision_backbone.fused_featurizer.blocks.19.norm1.bias": "model-00001-of-00004.safetensors",
800
+ "vision_backbone.fused_featurizer.blocks.19.norm1.weight": "model-00001-of-00004.safetensors",
801
+ "vision_backbone.fused_featurizer.blocks.19.norm2.bias": "model-00001-of-00004.safetensors",
802
+ "vision_backbone.fused_featurizer.blocks.19.norm2.weight": "model-00001-of-00004.safetensors",
803
+ "vision_backbone.fused_featurizer.blocks.2.attn.proj.bias": "model-00001-of-00004.safetensors",
804
+ "vision_backbone.fused_featurizer.blocks.2.attn.proj.weight": "model-00001-of-00004.safetensors",
805
+ "vision_backbone.fused_featurizer.blocks.2.attn.qkv.bias": "model-00001-of-00004.safetensors",
806
+ "vision_backbone.fused_featurizer.blocks.2.attn.qkv.weight": "model-00001-of-00004.safetensors",
807
+ "vision_backbone.fused_featurizer.blocks.2.mlp.fc1.bias": "model-00001-of-00004.safetensors",
808
+ "vision_backbone.fused_featurizer.blocks.2.mlp.fc1.weight": "model-00001-of-00004.safetensors",
809
+ "vision_backbone.fused_featurizer.blocks.2.mlp.fc2.bias": "model-00001-of-00004.safetensors",
810
+ "vision_backbone.fused_featurizer.blocks.2.mlp.fc2.weight": "model-00001-of-00004.safetensors",
811
+ "vision_backbone.fused_featurizer.blocks.2.norm1.bias": "model-00001-of-00004.safetensors",
812
+ "vision_backbone.fused_featurizer.blocks.2.norm1.weight": "model-00001-of-00004.safetensors",
813
+ "vision_backbone.fused_featurizer.blocks.2.norm2.bias": "model-00001-of-00004.safetensors",
814
+ "vision_backbone.fused_featurizer.blocks.2.norm2.weight": "model-00001-of-00004.safetensors",
815
+ "vision_backbone.fused_featurizer.blocks.20.attn.proj.bias": "model-00001-of-00004.safetensors",
816
+ "vision_backbone.fused_featurizer.blocks.20.attn.proj.weight": "model-00001-of-00004.safetensors",
817
+ "vision_backbone.fused_featurizer.blocks.20.attn.qkv.bias": "model-00001-of-00004.safetensors",
818
+ "vision_backbone.fused_featurizer.blocks.20.attn.qkv.weight": "model-00001-of-00004.safetensors",
819
+ "vision_backbone.fused_featurizer.blocks.20.mlp.fc1.bias": "model-00001-of-00004.safetensors",
820
+ "vision_backbone.fused_featurizer.blocks.20.mlp.fc1.weight": "model-00001-of-00004.safetensors",
821
+ "vision_backbone.fused_featurizer.blocks.20.mlp.fc2.bias": "model-00001-of-00004.safetensors",
822
+ "vision_backbone.fused_featurizer.blocks.20.mlp.fc2.weight": "model-00001-of-00004.safetensors",
823
+ "vision_backbone.fused_featurizer.blocks.20.norm1.bias": "model-00001-of-00004.safetensors",
824
+ "vision_backbone.fused_featurizer.blocks.20.norm1.weight": "model-00001-of-00004.safetensors",
825
+ "vision_backbone.fused_featurizer.blocks.20.norm2.bias": "model-00001-of-00004.safetensors",
826
+ "vision_backbone.fused_featurizer.blocks.20.norm2.weight": "model-00001-of-00004.safetensors",
827
+ "vision_backbone.fused_featurizer.blocks.21.attn.proj.bias": "model-00001-of-00004.safetensors",
828
+ "vision_backbone.fused_featurizer.blocks.21.attn.proj.weight": "model-00001-of-00004.safetensors",
829
+ "vision_backbone.fused_featurizer.blocks.21.attn.qkv.bias": "model-00001-of-00004.safetensors",
830
+ "vision_backbone.fused_featurizer.blocks.21.attn.qkv.weight": "model-00001-of-00004.safetensors",
831
+ "vision_backbone.fused_featurizer.blocks.21.mlp.fc1.bias": "model-00001-of-00004.safetensors",
832
+ "vision_backbone.fused_featurizer.blocks.21.mlp.fc1.weight": "model-00001-of-00004.safetensors",
833
+ "vision_backbone.fused_featurizer.blocks.21.mlp.fc2.bias": "model-00001-of-00004.safetensors",
834
+ "vision_backbone.fused_featurizer.blocks.21.mlp.fc2.weight": "model-00001-of-00004.safetensors",
835
+ "vision_backbone.fused_featurizer.blocks.21.norm1.bias": "model-00001-of-00004.safetensors",
836
+ "vision_backbone.fused_featurizer.blocks.21.norm1.weight": "model-00001-of-00004.safetensors",
837
+ "vision_backbone.fused_featurizer.blocks.21.norm2.bias": "model-00001-of-00004.safetensors",
838
+ "vision_backbone.fused_featurizer.blocks.21.norm2.weight": "model-00001-of-00004.safetensors",
839
+ "vision_backbone.fused_featurizer.blocks.22.attn.proj.bias": "model-00001-of-00004.safetensors",
840
+ "vision_backbone.fused_featurizer.blocks.22.attn.proj.weight": "model-00001-of-00004.safetensors",
841
+ "vision_backbone.fused_featurizer.blocks.22.attn.qkv.bias": "model-00001-of-00004.safetensors",
842
+ "vision_backbone.fused_featurizer.blocks.22.attn.qkv.weight": "model-00001-of-00004.safetensors",
843
+ "vision_backbone.fused_featurizer.blocks.22.mlp.fc1.bias": "model-00001-of-00004.safetensors",
844
+ "vision_backbone.fused_featurizer.blocks.22.mlp.fc1.weight": "model-00001-of-00004.safetensors",
845
+ "vision_backbone.fused_featurizer.blocks.22.mlp.fc2.bias": "model-00001-of-00004.safetensors",
846
+ "vision_backbone.fused_featurizer.blocks.22.mlp.fc2.weight": "model-00001-of-00004.safetensors",
847
+ "vision_backbone.fused_featurizer.blocks.22.norm1.bias": "model-00001-of-00004.safetensors",
848
+ "vision_backbone.fused_featurizer.blocks.22.norm1.weight": "model-00001-of-00004.safetensors",
849
+ "vision_backbone.fused_featurizer.blocks.22.norm2.bias": "model-00001-of-00004.safetensors",
850
+ "vision_backbone.fused_featurizer.blocks.22.norm2.weight": "model-00001-of-00004.safetensors",
851
+ "vision_backbone.fused_featurizer.blocks.23.attn.proj.bias": "model-00001-of-00004.safetensors",
852
+ "vision_backbone.fused_featurizer.blocks.23.attn.proj.weight": "model-00001-of-00004.safetensors",
853
+ "vision_backbone.fused_featurizer.blocks.23.attn.qkv.bias": "model-00001-of-00004.safetensors",
854
+ "vision_backbone.fused_featurizer.blocks.23.attn.qkv.weight": "model-00001-of-00004.safetensors",
855
+ "vision_backbone.fused_featurizer.blocks.23.mlp.fc1.bias": "model-00001-of-00004.safetensors",
856
+ "vision_backbone.fused_featurizer.blocks.23.mlp.fc1.weight": "model-00001-of-00004.safetensors",
857
+ "vision_backbone.fused_featurizer.blocks.23.mlp.fc2.bias": "model-00001-of-00004.safetensors",
858
+ "vision_backbone.fused_featurizer.blocks.23.mlp.fc2.weight": "model-00001-of-00004.safetensors",
859
+ "vision_backbone.fused_featurizer.blocks.23.norm1.bias": "model-00001-of-00004.safetensors",
860
+ "vision_backbone.fused_featurizer.blocks.23.norm1.weight": "model-00001-of-00004.safetensors",
861
+ "vision_backbone.fused_featurizer.blocks.23.norm2.bias": "model-00001-of-00004.safetensors",
862
+ "vision_backbone.fused_featurizer.blocks.23.norm2.weight": "model-00001-of-00004.safetensors",
863
+ "vision_backbone.fused_featurizer.blocks.24.attn.proj.bias": "model-00001-of-00004.safetensors",
864
+ "vision_backbone.fused_featurizer.blocks.24.attn.proj.weight": "model-00001-of-00004.safetensors",
865
+ "vision_backbone.fused_featurizer.blocks.24.attn.qkv.bias": "model-00001-of-00004.safetensors",
866
+ "vision_backbone.fused_featurizer.blocks.24.attn.qkv.weight": "model-00001-of-00004.safetensors",
867
+ "vision_backbone.fused_featurizer.blocks.24.mlp.fc1.bias": "model-00001-of-00004.safetensors",
868
+ "vision_backbone.fused_featurizer.blocks.24.mlp.fc1.weight": "model-00001-of-00004.safetensors",
869
+ "vision_backbone.fused_featurizer.blocks.24.mlp.fc2.bias": "model-00001-of-00004.safetensors",
870
+ "vision_backbone.fused_featurizer.blocks.24.mlp.fc2.weight": "model-00001-of-00004.safetensors",
871
+ "vision_backbone.fused_featurizer.blocks.24.norm1.bias": "model-00001-of-00004.safetensors",
872
+ "vision_backbone.fused_featurizer.blocks.24.norm1.weight": "model-00001-of-00004.safetensors",
873
+ "vision_backbone.fused_featurizer.blocks.24.norm2.bias": "model-00001-of-00004.safetensors",
874
+ "vision_backbone.fused_featurizer.blocks.24.norm2.weight": "model-00001-of-00004.safetensors",
875
+ "vision_backbone.fused_featurizer.blocks.25.attn.proj.bias": "model-00001-of-00004.safetensors",
876
+ "vision_backbone.fused_featurizer.blocks.25.attn.proj.weight": "model-00001-of-00004.safetensors",
877
+ "vision_backbone.fused_featurizer.blocks.25.attn.qkv.bias": "model-00001-of-00004.safetensors",
878
+ "vision_backbone.fused_featurizer.blocks.25.attn.qkv.weight": "model-00001-of-00004.safetensors",
879
+ "vision_backbone.fused_featurizer.blocks.25.mlp.fc1.bias": "model-00001-of-00004.safetensors",
880
+ "vision_backbone.fused_featurizer.blocks.25.mlp.fc1.weight": "model-00001-of-00004.safetensors",
881
+ "vision_backbone.fused_featurizer.blocks.25.mlp.fc2.bias": "model-00001-of-00004.safetensors",
882
+ "vision_backbone.fused_featurizer.blocks.25.mlp.fc2.weight": "model-00001-of-00004.safetensors",
883
+ "vision_backbone.fused_featurizer.blocks.25.norm1.bias": "model-00001-of-00004.safetensors",
884
+ "vision_backbone.fused_featurizer.blocks.25.norm1.weight": "model-00001-of-00004.safetensors",
885
+ "vision_backbone.fused_featurizer.blocks.25.norm2.bias": "model-00001-of-00004.safetensors",
886
+ "vision_backbone.fused_featurizer.blocks.25.norm2.weight": "model-00001-of-00004.safetensors",
887
+ "vision_backbone.fused_featurizer.blocks.26.attn.proj.bias": "model-00001-of-00004.safetensors",
888
+ "vision_backbone.fused_featurizer.blocks.26.attn.proj.weight": "model-00001-of-00004.safetensors",
889
+ "vision_backbone.fused_featurizer.blocks.26.attn.qkv.bias": "model-00001-of-00004.safetensors",
890
+ "vision_backbone.fused_featurizer.blocks.26.attn.qkv.weight": "model-00001-of-00004.safetensors",
891
+ "vision_backbone.fused_featurizer.blocks.26.mlp.fc1.bias": "model-00001-of-00004.safetensors",
892
+ "vision_backbone.fused_featurizer.blocks.26.mlp.fc1.weight": "model-00001-of-00004.safetensors",
893
+ "vision_backbone.fused_featurizer.blocks.26.mlp.fc2.bias": "model-00001-of-00004.safetensors",
894
+ "vision_backbone.fused_featurizer.blocks.26.mlp.fc2.weight": "model-00001-of-00004.safetensors",
895
+ "vision_backbone.fused_featurizer.blocks.26.norm1.bias": "model-00001-of-00004.safetensors",
896
+ "vision_backbone.fused_featurizer.blocks.26.norm1.weight": "model-00001-of-00004.safetensors",
897
+ "vision_backbone.fused_featurizer.blocks.26.norm2.bias": "model-00001-of-00004.safetensors",
898
+ "vision_backbone.fused_featurizer.blocks.26.norm2.weight": "model-00001-of-00004.safetensors",
899
+ "vision_backbone.fused_featurizer.blocks.3.attn.proj.bias": "model-00001-of-00004.safetensors",
900
+ "vision_backbone.fused_featurizer.blocks.3.attn.proj.weight": "model-00001-of-00004.safetensors",
901
+ "vision_backbone.fused_featurizer.blocks.3.attn.qkv.bias": "model-00001-of-00004.safetensors",
902
+ "vision_backbone.fused_featurizer.blocks.3.attn.qkv.weight": "model-00001-of-00004.safetensors",
903
+ "vision_backbone.fused_featurizer.blocks.3.mlp.fc1.bias": "model-00001-of-00004.safetensors",
904
+ "vision_backbone.fused_featurizer.blocks.3.mlp.fc1.weight": "model-00001-of-00004.safetensors",
905
+ "vision_backbone.fused_featurizer.blocks.3.mlp.fc2.bias": "model-00001-of-00004.safetensors",
906
+ "vision_backbone.fused_featurizer.blocks.3.mlp.fc2.weight": "model-00001-of-00004.safetensors",
907
+ "vision_backbone.fused_featurizer.blocks.3.norm1.bias": "model-00001-of-00004.safetensors",
908
+ "vision_backbone.fused_featurizer.blocks.3.norm1.weight": "model-00001-of-00004.safetensors",
909
+ "vision_backbone.fused_featurizer.blocks.3.norm2.bias": "model-00001-of-00004.safetensors",
910
+ "vision_backbone.fused_featurizer.blocks.3.norm2.weight": "model-00001-of-00004.safetensors",
911
+ "vision_backbone.fused_featurizer.blocks.4.attn.proj.bias": "model-00001-of-00004.safetensors",
912
+ "vision_backbone.fused_featurizer.blocks.4.attn.proj.weight": "model-00001-of-00004.safetensors",
913
+ "vision_backbone.fused_featurizer.blocks.4.attn.qkv.bias": "model-00001-of-00004.safetensors",
914
+ "vision_backbone.fused_featurizer.blocks.4.attn.qkv.weight": "model-00001-of-00004.safetensors",
915
+ "vision_backbone.fused_featurizer.blocks.4.mlp.fc1.bias": "model-00001-of-00004.safetensors",
916
+ "vision_backbone.fused_featurizer.blocks.4.mlp.fc1.weight": "model-00001-of-00004.safetensors",
917
+ "vision_backbone.fused_featurizer.blocks.4.mlp.fc2.bias": "model-00001-of-00004.safetensors",
918
+ "vision_backbone.fused_featurizer.blocks.4.mlp.fc2.weight": "model-00001-of-00004.safetensors",
919
+ "vision_backbone.fused_featurizer.blocks.4.norm1.bias": "model-00001-of-00004.safetensors",
920
+ "vision_backbone.fused_featurizer.blocks.4.norm1.weight": "model-00001-of-00004.safetensors",
921
+ "vision_backbone.fused_featurizer.blocks.4.norm2.bias": "model-00001-of-00004.safetensors",
922
+ "vision_backbone.fused_featurizer.blocks.4.norm2.weight": "model-00001-of-00004.safetensors",
923
+ "vision_backbone.fused_featurizer.blocks.5.attn.proj.bias": "model-00001-of-00004.safetensors",
924
+ "vision_backbone.fused_featurizer.blocks.5.attn.proj.weight": "model-00001-of-00004.safetensors",
925
+ "vision_backbone.fused_featurizer.blocks.5.attn.qkv.bias": "model-00001-of-00004.safetensors",
926
+ "vision_backbone.fused_featurizer.blocks.5.attn.qkv.weight": "model-00001-of-00004.safetensors",
927
+ "vision_backbone.fused_featurizer.blocks.5.mlp.fc1.bias": "model-00001-of-00004.safetensors",
928
+ "vision_backbone.fused_featurizer.blocks.5.mlp.fc1.weight": "model-00001-of-00004.safetensors",
929
+ "vision_backbone.fused_featurizer.blocks.5.mlp.fc2.bias": "model-00001-of-00004.safetensors",
930
+ "vision_backbone.fused_featurizer.blocks.5.mlp.fc2.weight": "model-00001-of-00004.safetensors",
931
+ "vision_backbone.fused_featurizer.blocks.5.norm1.bias": "model-00001-of-00004.safetensors",
932
+ "vision_backbone.fused_featurizer.blocks.5.norm1.weight": "model-00001-of-00004.safetensors",
933
+ "vision_backbone.fused_featurizer.blocks.5.norm2.bias": "model-00001-of-00004.safetensors",
934
+ "vision_backbone.fused_featurizer.blocks.5.norm2.weight": "model-00001-of-00004.safetensors",
935
+ "vision_backbone.fused_featurizer.blocks.6.attn.proj.bias": "model-00001-of-00004.safetensors",
936
+ "vision_backbone.fused_featurizer.blocks.6.attn.proj.weight": "model-00001-of-00004.safetensors",
937
+ "vision_backbone.fused_featurizer.blocks.6.attn.qkv.bias": "model-00001-of-00004.safetensors",
938
+ "vision_backbone.fused_featurizer.blocks.6.attn.qkv.weight": "model-00001-of-00004.safetensors",
939
+ "vision_backbone.fused_featurizer.blocks.6.mlp.fc1.bias": "model-00001-of-00004.safetensors",
940
+ "vision_backbone.fused_featurizer.blocks.6.mlp.fc1.weight": "model-00001-of-00004.safetensors",
941
+ "vision_backbone.fused_featurizer.blocks.6.mlp.fc2.bias": "model-00001-of-00004.safetensors",
942
+ "vision_backbone.fused_featurizer.blocks.6.mlp.fc2.weight": "model-00001-of-00004.safetensors",
943
+ "vision_backbone.fused_featurizer.blocks.6.norm1.bias": "model-00001-of-00004.safetensors",
944
+ "vision_backbone.fused_featurizer.blocks.6.norm1.weight": "model-00001-of-00004.safetensors",
945
+ "vision_backbone.fused_featurizer.blocks.6.norm2.bias": "model-00001-of-00004.safetensors",
946
+ "vision_backbone.fused_featurizer.blocks.6.norm2.weight": "model-00001-of-00004.safetensors",
947
+ "vision_backbone.fused_featurizer.blocks.7.attn.proj.bias": "model-00001-of-00004.safetensors",
948
+ "vision_backbone.fused_featurizer.blocks.7.attn.proj.weight": "model-00001-of-00004.safetensors",
949
+ "vision_backbone.fused_featurizer.blocks.7.attn.qkv.bias": "model-00001-of-00004.safetensors",
950
+ "vision_backbone.fused_featurizer.blocks.7.attn.qkv.weight": "model-00001-of-00004.safetensors",
951
+ "vision_backbone.fused_featurizer.blocks.7.mlp.fc1.bias": "model-00001-of-00004.safetensors",
952
+ "vision_backbone.fused_featurizer.blocks.7.mlp.fc1.weight": "model-00001-of-00004.safetensors",
953
+ "vision_backbone.fused_featurizer.blocks.7.mlp.fc2.bias": "model-00001-of-00004.safetensors",
954
+ "vision_backbone.fused_featurizer.blocks.7.mlp.fc2.weight": "model-00001-of-00004.safetensors",
955
+ "vision_backbone.fused_featurizer.blocks.7.norm1.bias": "model-00001-of-00004.safetensors",
956
+ "vision_backbone.fused_featurizer.blocks.7.norm1.weight": "model-00001-of-00004.safetensors",
957
+ "vision_backbone.fused_featurizer.blocks.7.norm2.bias": "model-00001-of-00004.safetensors",
958
+ "vision_backbone.fused_featurizer.blocks.7.norm2.weight": "model-00001-of-00004.safetensors",
959
+ "vision_backbone.fused_featurizer.blocks.8.attn.proj.bias": "model-00001-of-00004.safetensors",
960
+ "vision_backbone.fused_featurizer.blocks.8.attn.proj.weight": "model-00001-of-00004.safetensors",
961
+ "vision_backbone.fused_featurizer.blocks.8.attn.qkv.bias": "model-00001-of-00004.safetensors",
962
+ "vision_backbone.fused_featurizer.blocks.8.attn.qkv.weight": "model-00001-of-00004.safetensors",
963
+ "vision_backbone.fused_featurizer.blocks.8.mlp.fc1.bias": "model-00001-of-00004.safetensors",
964
+ "vision_backbone.fused_featurizer.blocks.8.mlp.fc1.weight": "model-00001-of-00004.safetensors",
965
+ "vision_backbone.fused_featurizer.blocks.8.mlp.fc2.bias": "model-00001-of-00004.safetensors",
966
+ "vision_backbone.fused_featurizer.blocks.8.mlp.fc2.weight": "model-00001-of-00004.safetensors",
967
+ "vision_backbone.fused_featurizer.blocks.8.norm1.bias": "model-00001-of-00004.safetensors",
968
+ "vision_backbone.fused_featurizer.blocks.8.norm1.weight": "model-00001-of-00004.safetensors",
969
+ "vision_backbone.fused_featurizer.blocks.8.norm2.bias": "model-00001-of-00004.safetensors",
970
+ "vision_backbone.fused_featurizer.blocks.8.norm2.weight": "model-00001-of-00004.safetensors",
971
+ "vision_backbone.fused_featurizer.blocks.9.attn.proj.bias": "model-00001-of-00004.safetensors",
972
+ "vision_backbone.fused_featurizer.blocks.9.attn.proj.weight": "model-00001-of-00004.safetensors",
973
+ "vision_backbone.fused_featurizer.blocks.9.attn.qkv.bias": "model-00001-of-00004.safetensors",
974
+ "vision_backbone.fused_featurizer.blocks.9.attn.qkv.weight": "model-00001-of-00004.safetensors",
975
+ "vision_backbone.fused_featurizer.blocks.9.mlp.fc1.bias": "model-00001-of-00004.safetensors",
976
+ "vision_backbone.fused_featurizer.blocks.9.mlp.fc1.weight": "model-00001-of-00004.safetensors",
977
+ "vision_backbone.fused_featurizer.blocks.9.mlp.fc2.bias": "model-00001-of-00004.safetensors",
978
+ "vision_backbone.fused_featurizer.blocks.9.mlp.fc2.weight": "model-00001-of-00004.safetensors",
979
+ "vision_backbone.fused_featurizer.blocks.9.norm1.bias": "model-00001-of-00004.safetensors",
980
+ "vision_backbone.fused_featurizer.blocks.9.norm1.weight": "model-00001-of-00004.safetensors",
981
+ "vision_backbone.fused_featurizer.blocks.9.norm2.bias": "model-00001-of-00004.safetensors",
982
+ "vision_backbone.fused_featurizer.blocks.9.norm2.weight": "model-00001-of-00004.safetensors",
983
+ "vision_backbone.fused_featurizer.norm.bias": "model-00001-of-00004.safetensors",
984
+ "vision_backbone.fused_featurizer.norm.weight": "model-00001-of-00004.safetensors",
985
+ "vision_backbone.fused_featurizer.patch_embed.proj.bias": "model-00001-of-00004.safetensors",
986
+ "vision_backbone.fused_featurizer.patch_embed.proj.weight": "model-00001-of-00004.safetensors",
987
+ "vision_backbone.fused_featurizer.pos_embed": "model-00001-of-00004.safetensors"
988
+ }
989
+ }
modeling_prismatic.py ADDED
@@ -0,0 +1,1085 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ modeling_prismatic.py
3
+
4
+ Core HuggingFace-style PrismaticPreTrainedModel and PrismaticForConditionalGeneration class definitions.
5
+ Inherits from the default `transformers.PretrainedModel`. Meant to be standalone and self-contained,
6
+ but exactly replicate the logic in `prismatic.models.vlms.prismatic.py`.
7
+ """
8
+
9
+ import logging
10
+ from dataclasses import dataclass
11
+ from functools import partial
12
+ from typing import Any, Callable, ClassVar, Dict, List, Optional, Tuple, Union
13
+
14
+ import numpy as np
15
+ import timm
16
+ import tokenizers
17
+ import torch
18
+ import torch.nn as nn
19
+ import transformers
20
+ from timm.models.vision_transformer import LayerScale
21
+ from transformers import AutoModelForCausalLM, PretrainedConfig, PreTrainedModel
22
+ from transformers.modeling_outputs import ModelOutput
23
+
24
+ from prismatic.training.train_utils import (
25
+ get_current_action_mask,
26
+ get_next_actions_mask,
27
+ )
28
+ from prismatic.vla.constants import (
29
+ ACTION_DIM,
30
+ ACTION_PROPRIO_NORMALIZATION_TYPE,
31
+ ACTION_TOKEN_BEGIN_IDX,
32
+ IGNORE_INDEX,
33
+ NUM_ACTIONS_CHUNK,
34
+ STOP_INDEX,
35
+ NormalizationType,
36
+ )
37
+
38
+ from .configuration_prismatic import OpenVLAConfig, PrismaticConfig
39
+
40
+ # Set up logger
41
+ logger = logging.getLogger(__name__)
42
+
43
+
44
+ # === Utility Functions for Monkey-Patching ===
45
+ def unpack_tuple(fn: Callable[[Any], Tuple[Any]]) -> Callable[[Any], Any]:
46
+ def wrapper(*args: Any, **kwargs: Any) -> Any:
47
+ result = fn(*args, **kwargs)
48
+ return result[0] if isinstance(result, tuple) else result
49
+
50
+ return wrapper
51
+
52
+
53
+ # HF Transformers overwrites parameters with names containing `gamma`; we're going to patch VisionBackbone.LayerScale.
54
+ # =>> TIMM :: https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/vision_transformer.py#L109
55
+ # =>> Transformers :: https://github.com/huggingface/transformers/blob/main/src/transformers/modeling_utils.py#L3960
56
+ def _ls_new_forward(self, x: torch.Tensor) -> torch.Tensor:
57
+ return x.mul_(self.scale_factor) if self.inplace else x * self.scale_factor
58
+
59
+
60
+ def ls_apply_patch(ls_module: LayerScale):
61
+ ls_module.scale_factor = nn.Parameter(ls_module.gamma.clone())
62
+ ls_module.forward = _ls_new_forward.__get__(ls_module, LayerScale)
63
+ del ls_module.gamma
64
+
65
+
66
+ # === Prismatic Vision Backbone (nn.Module) Definitions (w/ Fused Backbone Support) ===
67
+ class PrismaticVisionBackbone(nn.Module):
68
+ """
69
+ Vision backbone for Prismatic models that handles image feature extraction.
70
+
71
+ Supports both single backbone (e.g., SigLIP) and fused backbone (e.g., SigLIP + DINOv2) configurations.
72
+ For fused backbones, features from both models are concatenated along the feature dimension.
73
+ """
74
+
75
+ def __init__(
76
+ self,
77
+ use_fused_vision_backbone: bool,
78
+ image_sizes: List[int],
79
+ timm_model_ids: List[str],
80
+ timm_override_act_layers: List[Optional[str]],
81
+ ) -> None:
82
+ """
83
+ Initialize the vision backbone.
84
+
85
+ Args:
86
+ use_fused_vision_backbone: Whether to use two backbones and fuse their features
87
+ image_sizes: List of image sizes for each backbone
88
+ timm_model_ids: List of TIMM model IDs to use for each backbone
89
+ timm_override_act_layers: List of activation layer overrides for each backbone
90
+ """
91
+ super().__init__()
92
+ self.use_fused_vision_backbone = use_fused_vision_backbone
93
+ self.num_images_in_input = 1 # Default value, can be overridden later
94
+
95
+ # Validate number of (fused) vision backbones
96
+ if len(timm_model_ids) > 2:
97
+ raise ValueError("Prismatic models only support up to 2 (fused) vision backbones!")
98
+
99
+ # Create primary featurizer
100
+ self.featurizer = self._create_featurizer(
101
+ model_id=timm_model_ids[0], img_size=image_sizes[0], act_layer=timm_override_act_layers[0]
102
+ )
103
+ self.embed_dim = self.featurizer.embed_dim
104
+
105
+ # Create secondary featurizer if using fused backbone
106
+ if self.use_fused_vision_backbone:
107
+ self.fused_featurizer = self._create_featurizer(
108
+ model_id=timm_model_ids[1], img_size=image_sizes[1], act_layer=timm_override_act_layers[1]
109
+ )
110
+ self.embed_dim += self.fused_featurizer.embed_dim
111
+
112
+ # Patch LayerScale modules for HF compatibility
113
+ self._patch_layer_scales()
114
+
115
+ def _create_featurizer(self, model_id: str, img_size: int, act_layer: Optional[str]) -> nn.Module:
116
+ """
117
+ Create a TIMM-based featurizer model with appropriate configurations.
118
+
119
+ Args:
120
+ model_id: The TIMM model ID to load
121
+ img_size: Input image size for the model
122
+ act_layer: Override for the activation layer type
123
+
124
+ Returns:
125
+ A configured featurizer model
126
+ """
127
+ featurizer = timm.create_model(
128
+ model_id,
129
+ pretrained=False,
130
+ num_classes=0,
131
+ img_size=img_size,
132
+ act_layer=act_layer,
133
+ )
134
+
135
+ # Monkey-patch the forward function to extract the second-to-last layer features
136
+ num_blocks = len(featurizer.blocks)
137
+ featurizer.forward = unpack_tuple(partial(featurizer.get_intermediate_layers, n={num_blocks - 2}))
138
+
139
+ return featurizer
140
+
141
+ def _patch_layer_scales(self) -> None:
142
+ """
143
+ Patch all LayerScale modules to be compatible with HF's parameter naming.
144
+
145
+ HF Transformers overwrites parameters with names containing 'gamma',
146
+ so we need to rename and modify the forward method.
147
+ """
148
+ # Patch primary featurizer
149
+ for module in self.featurizer.modules():
150
+ if isinstance(module, LayerScale):
151
+ ls_apply_patch(module)
152
+
153
+ # Patch secondary featurizer if it exists
154
+ if self.use_fused_vision_backbone:
155
+ for module in self.fused_featurizer.modules():
156
+ if isinstance(module, LayerScale):
157
+ ls_apply_patch(module)
158
+
159
+ def get_num_patches(self) -> int:
160
+ """
161
+ Returns the number of vision patches output by the vision backbone.
162
+
163
+ Returns:
164
+ Number of patches per image
165
+ """
166
+ return self.featurizer.patch_embed.num_patches
167
+
168
+ def get_num_images_in_input(self) -> int:
169
+ """
170
+ Returns the number of input images for the vision backbone.
171
+
172
+ Returns:
173
+ Number of images expected in the input
174
+ """
175
+ return self.num_images_in_input
176
+
177
+ def set_num_images_in_input(self, num_images_in_input: int) -> None:
178
+ """
179
+ Sets the number of input images for the vision backbone.
180
+
181
+ Args:
182
+ num_images_in_input: Number of images to expect in the input
183
+ """
184
+ self.num_images_in_input = num_images_in_input
185
+
186
+ def forward(self, pixel_values: torch.Tensor) -> torch.Tensor:
187
+ """
188
+ Implements the forward pass for the vision backbone.
189
+
190
+ If `self.use_fused_vision_backbone == True`, uses both SigLIP and DINOv2 transformers to extract visual features
191
+ (otherwise uses SigLIP only). Allows multi-image inputs (but only for fused vision backbone).
192
+
193
+ Args:
194
+ pixel_values (torch.Tensor): Pixels for input image(s), (B, C, H, W).
195
+ """
196
+ if self.num_images_in_input == 1:
197
+ if not self.use_fused_vision_backbone:
198
+ return self.featurizer(pixel_values)
199
+
200
+ # Split `pixel_values :: [bsz, 2 * 3, resolution, resolution]` =>> featurize =>> channel stack
201
+ img, img_fused = torch.split(pixel_values, [3, 3], dim=1)
202
+ patches, patches_fused = self.featurizer(img), self.fused_featurizer(img_fused)
203
+
204
+ return torch.cat([patches, patches_fused], dim=2)
205
+
206
+ else:
207
+ assert self.use_fused_vision_backbone, "Multi-image inputs require using fused backbone!"
208
+
209
+ # Split `pixel_values` into individual images (each with 6 channels: 3 for SigLIP + 3 for DINOv2)
210
+ images = torch.split(pixel_values, [6] * self.num_images_in_input, dim=1)
211
+
212
+ # Process each image and collect patches
213
+ all_patches = []
214
+ for img in images:
215
+ # Split each image further into two stacks of channels (each with 3 channels)
216
+ img_regular, img_fused = torch.split(img, [3, 3], dim=1)
217
+
218
+ # Get patches from both SigLIP and DINOv2 vision transformers
219
+ patches = self.featurizer(img_regular)
220
+ patches_fused = self.fused_featurizer(img_fused)
221
+
222
+ # Concatenate SigLIP and DINOv2 patches along the hidden dimension
223
+ combined_patches = torch.cat([patches, patches_fused], dim=2)
224
+ all_patches.append(combined_patches)
225
+
226
+ # Concatenate all patches along the patch dimension
227
+ return torch.cat(all_patches, dim=1)
228
+
229
+
230
+ # === Prismatic Projector (nn.Module) Definitions ===
231
+ class PrismaticProjector(nn.Module):
232
+ def __init__(self, use_fused_vision_backbone: bool, vision_dim: int, llm_dim: int) -> None:
233
+ super().__init__()
234
+ self.use_fused_vision_backbone = use_fused_vision_backbone
235
+ self.vision_dim, self.llm_dim = vision_dim, llm_dim
236
+
237
+ # Switch on `use_fused_vision_backbone` =>> use slightly different MLPs and projection factors!
238
+ if not self.use_fused_vision_backbone:
239
+ self.fc1 = nn.Linear(self.vision_dim, self.llm_dim, bias=True)
240
+ self.fc2 = nn.Linear(self.llm_dim, self.llm_dim, bias=True)
241
+ self.act_fn1 = nn.GELU()
242
+ else:
243
+ initial_projection_dim = 4 * vision_dim
244
+ self.fc1 = nn.Linear(self.vision_dim, initial_projection_dim, bias=True)
245
+ self.fc2 = nn.Linear(initial_projection_dim, self.llm_dim, bias=True)
246
+ self.fc3 = nn.Linear(self.llm_dim, self.llm_dim, bias=True)
247
+ self.act_fn1 = nn.GELU()
248
+ self.act_fn2 = nn.GELU()
249
+
250
+ def forward(self, img_patches: torch.Tensor) -> torch.Tensor:
251
+ if not self.use_fused_vision_backbone:
252
+ projected_features = self.fc1(img_patches)
253
+ projected_features = self.act_fn1(projected_features)
254
+ projected_features = self.fc2(projected_features)
255
+ else:
256
+ projected_features = self.fc1(img_patches)
257
+ projected_features = self.act_fn1(projected_features)
258
+ projected_features = self.fc2(projected_features)
259
+ projected_features = self.act_fn2(projected_features)
260
+ projected_features = self.fc3(projected_features)
261
+
262
+ return projected_features
263
+
264
+
265
+ # === Main HF Class Definitions ===
266
+ @dataclass
267
+ class PrismaticCausalLMOutputWithPast(ModelOutput):
268
+ """Base class for Prismatic casual (visually-conditioned) language model outputs; also exposes visual features."""
269
+
270
+ loss: Optional[torch.FloatTensor] = None
271
+ logits: torch.FloatTensor = None
272
+ past_key_values: Optional[Tuple[Tuple[torch.FloatTensor]]] = None
273
+ hidden_states: Optional[Tuple[torch.FloatTensor, ...]] = None
274
+ attentions: Optional[Tuple[torch.FloatTensor]] = None
275
+
276
+ # Additions for VLMs
277
+ projector_features: Optional[torch.FloatTensor] = None
278
+
279
+
280
+ class PrismaticPreTrainedModel(PreTrainedModel):
281
+ config_class: PretrainedConfig = PrismaticConfig
282
+ base_model_prefix: str = "model"
283
+ supports_gradient_checkpointing: bool = True
284
+
285
+ _no_split_modules: ClassVar[List[str]] = ["PrismaticProjector"]
286
+ _skip_keys_device_placement: str = "past_key_values"
287
+ _supports_flash_attn_2: bool = True
288
+
289
+ def _init_weights(self, module: nn.Module) -> None:
290
+ # Important :: this HF ported version is *not* meant for training from scratch; only inference and fine-tuning!
291
+ # => As such, this init_weights code is not correct; if training VLMs from scratch, use the main codebase at
292
+ # https://github.com/TRI-ML/prismatic-vlms
293
+ std = (
294
+ self.config.initializer_range
295
+ if hasattr(self.config, "initializer_range")
296
+ else self.config.text_config.initializer_range
297
+ )
298
+
299
+ if hasattr(module, "class_embedding"):
300
+ module.class_embedding.data.normal_(mean=0.0, std=std)
301
+
302
+ if isinstance(module, (nn.Linear, nn.Conv2d)):
303
+ module.weight.data.normal_(mean=0.0, std=std)
304
+ if module.bias is not None:
305
+ module.bias.data.zero_()
306
+ elif isinstance(module, nn.Embedding):
307
+ module.weight.data.normal_(mean=0.0, std=std)
308
+ if module.padding_idx is not None:
309
+ module.weight.data[module.padding_idx].zero_()
310
+
311
+ @property
312
+ def _supports_sdpa(self) -> bool:
313
+ """Check LLM supports SDPA Attention"""
314
+ return self.language_model._supports_sdpa
315
+
316
+
317
+ class PrismaticForConditionalGeneration(PrismaticPreTrainedModel):
318
+ def __init__(self, config: PrismaticConfig) -> None:
319
+ super().__init__(config)
320
+
321
+ # [Validation] Lightweight Validate on `config` Fields + Dependency Versions
322
+ if config.use_fused_vision_backbone is None:
323
+ raise ValueError("Missing config field `use_fused_vision_backbone`")
324
+
325
+ if timm.__version__ not in {"0.9.10", "0.9.11", "0.9.12", "0.9.16"}:
326
+ raise NotImplementedError(
327
+ "TIMM Version must be >= 0.9.10 and < 1.0.0 (breaking); please raise a GitHub Issue "
328
+ "if you urgently need support for latest TIMM versions."
329
+ )
330
+
331
+ if (transformers.__version__ != "4.40.1") or (tokenizers.__version__ != "0.19.1"):
332
+ logger.warning(
333
+ f"Expected `transformers==4.40.1` and `tokenizers==0.19.1` but got "
334
+ f"`transformers=={transformers.__version__}` and `tokenizers=={tokenizers.__version__}`; "
335
+ f"there might be inference-time regressions due to dependency changes. If in doubt, please"
336
+ f"use the above versions."
337
+ )
338
+
339
+ # Instantiate PrismaticVisionBackbone (w/ Potential Fused Backbone)
340
+ self.vision_backbone = PrismaticVisionBackbone(
341
+ config.use_fused_vision_backbone, config.image_sizes, config.timm_model_ids, config.timm_override_act_layers
342
+ )
343
+
344
+ # Create Multimodal Projector
345
+ self.projector = PrismaticProjector(
346
+ config.use_fused_vision_backbone,
347
+ vision_dim=self.vision_backbone.embed_dim,
348
+ llm_dim=config.text_config.hidden_size,
349
+ )
350
+
351
+ # Instantiate LLM Backbone
352
+ self.language_model = AutoModelForCausalLM.from_config(
353
+ config.text_config, attn_implementation=config._attn_implementation
354
+ )
355
+ self.vocab_size = config.text_config.vocab_size
356
+ self.pad_token_id = config.pad_token_id
357
+ self.llm_dim = config.text_config.hidden_size
358
+
359
+ # HF Boilerplate =>> initializes weights via `_init_weights()` and sets gradient checkpointing
360
+ self.post_init()
361
+
362
+ # === `PreTrainedModel` Boilerplate ===
363
+ def get_input_embeddings(self) -> nn.Module:
364
+ return self.language_model.get_input_embeddings()
365
+
366
+ def set_input_embeddings(self, value: nn.Module) -> None:
367
+ self.language_model.set_input_embeddings(value)
368
+
369
+ def get_output_embeddings(self) -> nn.Module:
370
+ return self.language_model.get_output_embeddings()
371
+
372
+ def set_output_embeddings(self, new_embeddings: nn.Module) -> None:
373
+ self.language_model.set_output_embeddings(new_embeddings)
374
+
375
+ def get_decoder(self) -> nn.Module:
376
+ return self.language_model.get_decoder()
377
+
378
+ def set_decoder(self, decoder: nn.Module) -> None:
379
+ self.language_model.set_decoder(decoder)
380
+
381
+ def tie_weights(self) -> None:
382
+ self.language_model.tie_weights() # Note: `Llama-2` and `Mistral` don't tie weights (no-op)
383
+
384
+ def resize_token_embeddings(
385
+ self, new_num_tokens: Optional[int] = None, pad_to_multiple_of: Optional[int] = None
386
+ ) -> nn.Embedding:
387
+ updated_embeddings = self.language_model.resize_token_embeddings(new_num_tokens, pad_to_multiple_of)
388
+
389
+ # Update config/instance variables
390
+ self.config.text_config.vocab_size = updated_embeddings.num_embeddings
391
+ self.vocab_size = updated_embeddings.num_embeddings
392
+
393
+ return updated_embeddings
394
+
395
+ def _replace_input_embeddings(self, input_embeddings, all_actions_mask, noisy_action_features):
396
+ """
397
+ Replace embeddings in input_embeddings at positions where all_actions_mask is True
398
+ with embeddings from noisy_action_features, using vectorized operations.
399
+
400
+ Args:
401
+ input_embeddings: Tensor of shape (B, S, D)
402
+ all_actions_mask: Boolean tensor of shape (B, S)
403
+ noisy_action_features: Tensor of shape (B, K, D) where K is the number of True values in mask per sample
404
+
405
+ Returns:
406
+ Modified input_embeddings tensor
407
+ """
408
+ # Clone input to avoid modifying the original tensor
409
+ new_input_embeddings = input_embeddings.clone()
410
+
411
+ # Create a tensor with the same shape of input_embeddings to hold the noisy action features
412
+ repositioned_noisy_action_features = torch.zeros_like(input_embeddings)
413
+
414
+ # Create batch indices for splicing
415
+ batch_indices = torch.arange(input_embeddings.shape[0], device=input_embeddings.device)
416
+ batch_indices = batch_indices.unsqueeze(1).expand(-1, noisy_action_features.shape[1])
417
+
418
+ # Get indices where mask is True for each sample
419
+ masked_indices = torch.stack([torch.where(mask)[0] for mask in all_actions_mask])
420
+
421
+ # Move the noisy action features into their correct positions
422
+ repositioned_noisy_action_features[batch_indices, masked_indices] = noisy_action_features
423
+
424
+ # Combine original input embeddings and noisy action embeddings using the mask
425
+ new_input_embeddings = torch.where(
426
+ all_actions_mask.unsqueeze(-1), repositioned_noisy_action_features, new_input_embeddings
427
+ )
428
+
429
+ return new_input_embeddings
430
+
431
+ def _process_action_masks(self, labels):
432
+ """Helper to get action masks from labels"""
433
+ current_action_mask = get_current_action_mask(labels)
434
+ next_actions_mask = get_next_actions_mask(labels)
435
+ all_actions_mask = current_action_mask | next_actions_mask # (B, seq_len)
436
+ return all_actions_mask
437
+
438
+ def _process_vision_features(self, pixel_values, language_embeddings=None, use_film=False):
439
+ """Process vision features with optional FiLM conditioning"""
440
+ if use_film:
441
+ # FiLM: Infuse language inputs into visual features
442
+ patch_features = self.vision_backbone(pixel_values, language_embeddings) # (bsz, 256 * num_images, D)
443
+ else:
444
+ patch_features = self.vision_backbone(pixel_values) # (bsz, 256 * num_images, D)
445
+
446
+ # Project patch embeddings into language embedding space
447
+ return self.projector(patch_features)
448
+
449
+ def _process_proprio_features(self, projected_patch_embeddings, proprio, proprio_projector):
450
+ """Process proprioceptive features and append to vision features"""
451
+ if proprio_projector is not None and proprio is not None:
452
+ # projected_patch_embeddings: (bsz, num_patches * num_images, llm_dim)
453
+ # proprio: (bsz, proprio_dim) or (propro_dim,)
454
+ proprio = proprio.reshape(projected_patch_embeddings.shape[0], -1) # (bsz, proprio_dim)
455
+ proprio_features = proprio_projector(proprio) # (bsz, llm_dim)
456
+ proprio_features = proprio_features.unsqueeze(dim=1) # (bsz, 1, llm_dim)
457
+ # For simplicity, just append proprio token to the end of projected vision patch tokens
458
+ return torch.cat((projected_patch_embeddings, proprio_features), dim=1)
459
+ return projected_patch_embeddings
460
+
461
+ def _build_multimodal_attention(self, input_embeddings, projected_patch_embeddings, attention_mask):
462
+ """Build multimodal embeddings and attention mask"""
463
+ # Update attention mask
464
+ projected_patch_attention_mask = None
465
+ if attention_mask is not None:
466
+ projected_patch_attention_mask = torch.full(
467
+ (projected_patch_embeddings.shape[0], projected_patch_embeddings.shape[1]),
468
+ fill_value=True,
469
+ dtype=attention_mask.dtype,
470
+ device=attention_mask.device,
471
+ )
472
+
473
+ # Build multimodal embeddings & attention mask; insert embeddings after <BOS> token (1:)
474
+ multimodal_embeddings = torch.cat(
475
+ [input_embeddings[:, :1, :], projected_patch_embeddings, input_embeddings[:, 1:, :]], dim=1
476
+ )
477
+
478
+ multimodal_attention_mask = None
479
+ if attention_mask is not None:
480
+ multimodal_attention_mask = torch.cat(
481
+ [attention_mask[:, :1], projected_patch_attention_mask, attention_mask[:, 1:]], dim=1
482
+ )
483
+
484
+ return multimodal_embeddings, multimodal_attention_mask
485
+
486
+ def _build_multimodal_labels(self, labels, projected_patch_embeddings):
487
+ """Build multimodal labels with IGNORE_INDEX for patch embeddings"""
488
+ if labels is not None:
489
+ projected_patch_labels = torch.full(
490
+ (projected_patch_embeddings.shape[0], projected_patch_embeddings.shape[1]),
491
+ fill_value=IGNORE_INDEX,
492
+ dtype=labels.dtype,
493
+ device=labels.device,
494
+ )
495
+ return torch.cat([labels[:, :1], projected_patch_labels, labels[:, 1:]], dim=1)
496
+ return None
497
+
498
+ # === Core Prismatic VLM `forward()` Logic ===
499
+ def forward(
500
+ self,
501
+ input_ids: Optional[torch.LongTensor] = None,
502
+ attention_mask: Optional[torch.Tensor] = None,
503
+ pixel_values: Optional[torch.FloatTensor] = None,
504
+ labels: Optional[torch.LongTensor] = None,
505
+ inputs_embeds: Optional[torch.FloatTensor] = None,
506
+ past_key_values: Optional[List[torch.FloatTensor]] = None,
507
+ use_cache: Optional[bool] = None,
508
+ output_attentions: Optional[bool] = None,
509
+ output_hidden_states: Optional[bool] = None,
510
+ output_projector_features: Optional[bool] = None,
511
+ return_dict: Optional[bool] = None,
512
+ proprio=None,
513
+ proprio_projector=None,
514
+ noisy_actions=None,
515
+ noisy_action_projector=None,
516
+ diffusion_timestep_embeddings=None,
517
+ use_film: bool = False,
518
+ ) -> Union[Tuple, PrismaticCausalLMOutputWithPast]:
519
+ """Run a forward pass through the VLM, returning a PrismaticCausalLMOutputWithPast instance."""
520
+ output_attentions = output_attentions if output_attentions is not None else self.config.output_attentions
521
+ output_hidden_states = (
522
+ output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states
523
+ )
524
+ output_projector_features = output_projector_features if output_projector_features is not None else False
525
+ return_dict = return_dict if return_dict is not None else self.config.use_return_dict
526
+
527
+ # Respect `use_cache` only if not training (even if `gradient_checkpointing` is off)
528
+ use_cache = use_cache and not self.training
529
+
530
+ # Instantiate Placeholder for Projector Features
531
+ projected_patch_embeddings = None
532
+
533
+ # === Handle Generation with Cache (`input_ids.shape[1] == 1`) =>> requires `past_keys_values` ===
534
+ if input_ids.shape[1] == 1:
535
+ assert input_ids.shape[0] == 1, "Generation is only currently supported for batch size of 1!"
536
+ assert past_key_values is not None, "You must provide `past_key_values` during cached generation!"
537
+ assert labels is None, "Unexpected key `labels` provided during cached generation!"
538
+
539
+ language_model_output = self.language_model(
540
+ input_ids=input_ids,
541
+ attention_mask=None,
542
+ position_ids=None,
543
+ past_key_values=past_key_values,
544
+ inputs_embeds=None,
545
+ labels=None,
546
+ use_cache=use_cache,
547
+ output_attentions=output_attentions,
548
+ output_hidden_states=output_hidden_states,
549
+ return_dict=return_dict,
550
+ )
551
+
552
+ # === Handle Unimodal Forward ===
553
+ elif pixel_values is None:
554
+ assert (input_ids is not None) and (inputs_embeds is None), "Missing `input_ids` in language-only forward!"
555
+ assert past_key_values is None, "Unexpected key `past_key_values` provided during language-only forward!"
556
+
557
+ language_model_output = self.language_model(
558
+ input_ids=input_ids,
559
+ attention_mask=attention_mask,
560
+ position_ids=None,
561
+ past_key_values=None,
562
+ inputs_embeds=None,
563
+ labels=labels,
564
+ use_cache=use_cache,
565
+ output_attentions=output_attentions,
566
+ output_hidden_states=output_hidden_states,
567
+ return_dict=return_dict,
568
+ )
569
+
570
+ # === Handle Multimodal Forward ===
571
+ elif (input_ids.shape[0] == pixel_values.shape[0]) or (inputs_embeds.shape[0] == pixel_values.shape[0]):
572
+ assert past_key_values is None, "Unexpected key `past_key_values` provided during multimodal forward!"
573
+
574
+ # Get input embeddings (from language model embeddings)
575
+ input_embeddings = self.get_input_embeddings()(input_ids) # (B, seq_len, D)
576
+
577
+ # Extract action masks
578
+ all_actions_mask = self._process_action_masks(labels)
579
+
580
+ # Extract the language portion of the input embeddings (i.e. remove the action tokens portion)
581
+ language_embeddings = input_embeddings[~all_actions_mask].reshape(
582
+ input_embeddings.shape[0], -1, input_embeddings.shape[2]
583
+ ) # (B, lang_seq_len, llm_dim)
584
+
585
+ # Get visual features
586
+ projected_patch_embeddings = self._process_vision_features(pixel_values, language_embeddings, use_film)
587
+
588
+ # Add proprioceptive state if provided
589
+ projected_patch_embeddings = self._process_proprio_features(
590
+ projected_patch_embeddings, proprio, proprio_projector
591
+ )
592
+
593
+ # [Diffusion] Add diffusion timestep embedding if provided
594
+ if diffusion_timestep_embeddings is not None:
595
+ # For simplicity, just append diffusion timestep embedding to the end of projected vision patch tokens
596
+ projected_patch_embeddings = torch.cat(
597
+ (projected_patch_embeddings, diffusion_timestep_embeddings), dim=1
598
+ )
599
+
600
+ # Process action embeddings
601
+ if noisy_actions is not None:
602
+ # Get mask corresponding to all action tokens
603
+ all_actions_mask = self._process_action_masks(labels)
604
+
605
+ # Reshape noisy actions into individual action tokens
606
+ # noisy_actions: (B, chunk_len, action_dim) -> (B, chunk_len * action_dim, 1)
607
+ B = noisy_actions.shape[0]
608
+ noisy_actions = noisy_actions.reshape(B, -1).unsqueeze(-1)
609
+
610
+ # Project noisy action tokens into language model embedding space
611
+ noisy_action_features = noisy_action_projector(noisy_actions) # (B, chunk_len * action_dim, llm_dim)
612
+
613
+ # Replace embeddings of the action tokens with noisy action embeddings
614
+ input_embeddings = self._replace_input_embeddings(
615
+ input_embeddings, all_actions_mask, noisy_action_features
616
+ )
617
+ else:
618
+ # Replace the embeddings of the action tokens with zeros
619
+ # (Later on, the positional embeddings will be added to them)
620
+ all_actions_mask = all_actions_mask.unsqueeze(-1) # (B, seq_len, 1)
621
+ input_embeddings = input_embeddings * ~all_actions_mask
622
+
623
+ # Build multimodal embeddings & attention mask
624
+ multimodal_embeddings, multimodal_attention_mask = self._build_multimodal_attention(
625
+ input_embeddings, projected_patch_embeddings, attention_mask
626
+ )
627
+
628
+ # Build labels for multimodal sequence if needed
629
+ multimodal_labels = self._build_multimodal_labels(labels, projected_patch_embeddings)
630
+
631
+ # Dispatch to language model
632
+ language_model_output = self.language_model(
633
+ input_ids=None,
634
+ attention_mask=multimodal_attention_mask,
635
+ position_ids=None,
636
+ past_key_values=None,
637
+ inputs_embeds=multimodal_embeddings,
638
+ labels=multimodal_labels,
639
+ use_cache=use_cache,
640
+ output_attentions=output_attentions,
641
+ output_hidden_states=output_hidden_states,
642
+ return_dict=return_dict,
643
+ )
644
+
645
+ # === Otherwise =>> Assume Invalid! ===
646
+ elif (input_ids.shape[0] != pixel_values.shape[0]) or (inputs_embeds.shape[0] != pixel_values.shape[0]):
647
+ raise ValueError("Non-homogenous batch of (text, image) input -- forward() does not support mixed batches!")
648
+
649
+ else:
650
+ raise ValueError(
651
+ "Invalid PrismaticForConditionalGeneration `forward()` call with provided arguments:\n"
652
+ f"=> `input_ids` = {input_ids is not None}\n"
653
+ f"=> `attention_mask` = {attention_mask is not None}\n"
654
+ f"=> `pixel_values` = {pixel_values is not None}\n"
655
+ f"=> `labels` = {labels is not None}\n"
656
+ f"=> `input_embeds` = {inputs_embeds is not None}\n"
657
+ f"=> `past_key_values` = {past_key_values is not None}\n"
658
+ f"=> `use_cache` = {use_cache}"
659
+ )
660
+
661
+ # Unpack `language_model_output` and return PrismaticCausalLMOutputWithPast (or tuple if not `return_dict`)
662
+ if not return_dict:
663
+ if output_projector_features and (projected_patch_embeddings is not None):
664
+ return *language_model_output, projected_patch_embeddings
665
+
666
+ return language_model_output
667
+
668
+ return PrismaticCausalLMOutputWithPast(
669
+ loss=language_model_output.loss,
670
+ logits=language_model_output.logits,
671
+ past_key_values=language_model_output.past_key_values,
672
+ hidden_states=language_model_output.hidden_states,
673
+ attentions=language_model_output.attentions,
674
+ projector_features=projected_patch_embeddings,
675
+ )
676
+
677
+ # === GenerationMixin Methods ===
678
+ def prepare_inputs_for_generation(
679
+ self,
680
+ input_ids: Optional[torch.Tensor] = None,
681
+ past_key_values: Optional[List[torch.FloatTensor]] = None,
682
+ inputs_embeds: Optional[torch.FloatTensor] = None,
683
+ pixel_values: Optional[torch.FloatTensor] = None,
684
+ attention_mask: Optional[torch.Tensor] = None,
685
+ **kwargs: str,
686
+ ) -> Dict[str, torch.Tensor]:
687
+ """Borrowed from `LlamaForCausalLM` and simplified for batch size = 1; mirrors original PrismaticVLM logic."""
688
+ if ((input_ids is not None) and (input_ids.shape[0] > 1)) or (
689
+ (inputs_embeds is not None) and (inputs_embeds.shape[0] > 1)
690
+ ):
691
+ raise ValueError("Generation with batch size > 1 is not currently supported!")
692
+
693
+ # Handle `past_key_values` (cache) =>> assume `input_ids` just has unprocessed tokens
694
+ if past_key_values is not None:
695
+ input_ids = input_ids[:, -1:]
696
+
697
+ # If `input_embeds` are passed, we only want to use them in the 1st generation step
698
+ if inputs_embeds is not None and past_key_values is None:
699
+ model_inputs = {"input_embeds": inputs_embeds}
700
+ else:
701
+ model_inputs = {"input_ids": input_ids}
702
+
703
+ # Make sure `pixel_values` are preserved in `model_inputs`
704
+ model_inputs.update(
705
+ {
706
+ "attention_mask": attention_mask,
707
+ "pixel_values": pixel_values,
708
+ "past_key_values": past_key_values,
709
+ "use_cache": kwargs.get("use_cache"),
710
+ }
711
+ )
712
+
713
+ return model_inputs
714
+
715
+ # Defer to Language Model (all handle this differently, with different return types)
716
+ def _reorder_cache(self, *args, **kwargs) -> Any:
717
+ return self.language_model._reorder_cache(*args, **kwargs)
718
+
719
+
720
+ class OpenVLAForActionPrediction(PrismaticForConditionalGeneration):
721
+ config_class: PretrainedConfig = OpenVLAConfig
722
+
723
+ def __init__(self, config: OpenVLAConfig) -> None:
724
+ super().__init__(config)
725
+ self.norm_stats = config.norm_stats
726
+
727
+ # Compute action bins
728
+ self.bins = np.linspace(-1, 1, config.n_action_bins)
729
+ self.bin_centers = (self.bins[:-1] + self.bins[1:]) / 2.0
730
+
731
+ # Compute vocab size for de-tokenization -- revert added "multiple of"
732
+ self.vocab_size = self.config.text_config.vocab_size - self.config.pad_to_multiple_of
733
+
734
+ def _prepare_input_for_action_prediction(self, input_ids, attention_mask):
735
+ """Prepares input for action prediction by adding necessary tokens"""
736
+ # Add (ACTION_DIM * NUM_ACTIONS_CHUNK) placeholder tokens to input_ids to simulate action tokens
737
+ placeholder_action_token_ids = (
738
+ torch.ones((input_ids.shape[0], ACTION_DIM * NUM_ACTIONS_CHUNK)).to(input_ids.device).to(input_ids.dtype)
739
+ )
740
+ input_ids = torch.cat([input_ids, placeholder_action_token_ids], dim=-1)
741
+
742
+ # Add stop token to sequence (needed in non-causal bi-directional self-attention, as it appears at train time)
743
+ stop_token_id = torch.ones((input_ids.shape[0], 1)).to(input_ids.device).to(input_ids.dtype) * STOP_INDEX
744
+ input_ids = torch.cat([input_ids, stop_token_id], dim=-1)
745
+
746
+ # Extend the attention mask to fit the new shape of input
747
+ # Note: Only batch size == 1 supported right now
748
+ mask_extension = (
749
+ torch.ones((attention_mask.shape[0], input_ids.shape[-1] - attention_mask.shape[-1]))
750
+ .to(attention_mask.device)
751
+ .to(attention_mask.dtype)
752
+ )
753
+ attention_mask = torch.cat([attention_mask, mask_extension], dim=-1)
754
+
755
+ return input_ids, attention_mask
756
+
757
+ def _prepare_labels_for_action_prediction(self, labels, input_ids):
758
+ """Creates labels tensor for action prediction if not provided"""
759
+ # Extend labels tensor with fake action labels
760
+ ARBITRARY_ACTION_TOKEN_IDX = ACTION_TOKEN_BEGIN_IDX + 1
761
+ labels_extension = (
762
+ torch.ones((labels.shape[0], input_ids.shape[-1] - labels.shape[-1])).to(labels.device).to(labels.dtype)
763
+ * ARBITRARY_ACTION_TOKEN_IDX
764
+ )
765
+ labels = torch.cat([labels, labels_extension], dim=-1)
766
+
767
+ # Replace last label token with stop token
768
+ labels[:, -1] = STOP_INDEX
769
+
770
+ return labels
771
+
772
+ def _unnormalize_actions(self, normalized_actions, unnorm_key=None):
773
+ """Unnormalize actions using dataset statistics"""
774
+ action_norm_stats = self.get_action_stats(unnorm_key)
775
+
776
+ if ACTION_PROPRIO_NORMALIZATION_TYPE == NormalizationType.BOUNDS:
777
+ mask = action_norm_stats.get("mask", np.ones_like(action_norm_stats["min"], dtype=bool))
778
+ action_high, action_low = np.array(action_norm_stats["max"]), np.array(action_norm_stats["min"])
779
+ elif ACTION_PROPRIO_NORMALIZATION_TYPE == NormalizationType.BOUNDS_Q99:
780
+ mask = action_norm_stats.get("mask", np.ones_like(action_norm_stats["q01"], dtype=bool))
781
+ action_high, action_low = np.array(action_norm_stats["q99"]), np.array(action_norm_stats["q01"])
782
+ else:
783
+ raise ValueError("Unsupported action/proprio normalization type detected!")
784
+
785
+ actions = np.where(
786
+ mask,
787
+ 0.5 * (normalized_actions + 1) * (action_high - action_low + 1e-8) + action_low,
788
+ normalized_actions,
789
+ )
790
+
791
+ return actions
792
+
793
+ def _run_diffusion_prediction(
794
+ self,
795
+ input_embeddings,
796
+ all_actions_mask,
797
+ noise,
798
+ action_head,
799
+ projected_patch_embeddings,
800
+ labels,
801
+ attention_mask,
802
+ NUM_PATCHES,
803
+ NUM_PROMPT_TOKENS,
804
+ noisy_action_projector,
805
+ ):
806
+ """Run diffusion-based action prediction"""
807
+ # Clone embedding for reuse in each timestep
808
+ orig_projected_patch_embeddings = projected_patch_embeddings.clone()
809
+ curr_noisy_actions = noise
810
+
811
+ # Reverse diffusion: Iteratively denoise to generate action prediction
812
+ for t in action_head.noise_scheduler.timesteps:
813
+ # Get diffusion model's noise prediction (conditioned on VLA latent embedding, current noisy action
814
+ # embedding, and diffusion timestep embedding)
815
+ timesteps = torch.Tensor([t]).to(labels.device)
816
+ diffusion_timestep_embeddings = (
817
+ action_head.time_encoder(timesteps).to(curr_noisy_actions.dtype).to(curr_noisy_actions.device)
818
+ ) # (B, llm_dim)
819
+ diffusion_timestep_embeddings = diffusion_timestep_embeddings.unsqueeze(1) # (B, 1, llm_dim)
820
+
821
+ # [Diffusion] Replace the embeddings of the action tokens with noisy actions
822
+ # (Later on, the positional embeddings will be added to them)
823
+
824
+ # For simplicity, append diffusion timestep embedding to the end of projected vision tokens
825
+ projected_patch_embeddings = torch.cat(
826
+ (orig_projected_patch_embeddings, diffusion_timestep_embeddings), dim=1
827
+ )
828
+
829
+ # Reshape and project noisy actions into language embedding space
830
+ B = curr_noisy_actions.shape[0]
831
+ orig_curr_noisy_actions_shape = curr_noisy_actions.shape
832
+ curr_noisy_actions = curr_noisy_actions.reshape(B, -1).unsqueeze(-1)
833
+ noisy_action_features = noisy_action_projector(curr_noisy_actions)
834
+ curr_noisy_actions = curr_noisy_actions.reshape(orig_curr_noisy_actions_shape)
835
+
836
+ # Replace action token embeddings with noisy action embeddings
837
+ input_embeddings = self._replace_input_embeddings(
838
+ input_embeddings.clone(), all_actions_mask, noisy_action_features
839
+ )
840
+
841
+ # Build multimodal embeddings and attention mask
842
+ multimodal_embeddings, multimodal_attention_mask = self._build_multimodal_attention(
843
+ input_embeddings, projected_patch_embeddings, attention_mask
844
+ )
845
+
846
+ # Forward pass through language model
847
+ language_model_output = self.language_model(
848
+ input_ids=None,
849
+ attention_mask=multimodal_attention_mask,
850
+ position_ids=None,
851
+ past_key_values=None,
852
+ inputs_embeds=multimodal_embeddings,
853
+ labels=None,
854
+ use_cache=None,
855
+ output_attentions=False,
856
+ output_hidden_states=True,
857
+ return_dict=True,
858
+ )
859
+
860
+ # Extract hidden states for action portion of response
861
+ last_hidden_states = language_model_output.hidden_states[-1] # (B, seq_len, D)
862
+ actions_hidden_states = last_hidden_states[
863
+ :,
864
+ NUM_PATCHES + NUM_PROMPT_TOKENS : NUM_PATCHES + NUM_PROMPT_TOKENS + ACTION_DIM * NUM_ACTIONS_CHUNK,
865
+ :,
866
+ ] # (B, act_chunk_len, D)
867
+
868
+ # Predict noise and update noisy actions: x_t -> x_{t-1}
869
+ noise_pred = action_head.predict_noise(actions_hidden_states)
870
+ curr_noisy_actions = action_head.noise_scheduler.step(noise_pred, t, curr_noisy_actions).prev_sample
871
+
872
+ curr_noisy_actions = curr_noisy_actions.reshape(NUM_ACTIONS_CHUNK, ACTION_DIM)
873
+
874
+ # Return final actions
875
+ return curr_noisy_actions.float().cpu().detach().numpy(), actions_hidden_states
876
+
877
+ def _regression_or_discrete_prediction(
878
+ self,
879
+ input_embeddings,
880
+ all_actions_mask,
881
+ projected_patch_embeddings,
882
+ attention_mask,
883
+ labels,
884
+ NUM_PATCHES,
885
+ NUM_PROMPT_TOKENS,
886
+ action_head=None,
887
+ ):
888
+ """Run L1 regression-based continuous action prediction or discrete action tokens prediction."""
889
+ # Zero out action token embeddings
890
+ all_actions_mask = all_actions_mask.unsqueeze(-1) # (B, seq_len, 1)
891
+ input_embeddings = input_embeddings * ~all_actions_mask
892
+
893
+ # Build multimodal embeddings and attention mask
894
+ multimodal_embeddings, multimodal_attention_mask = self._build_multimodal_attention(
895
+ input_embeddings, projected_patch_embeddings, attention_mask
896
+ )
897
+
898
+ # Forward pass through language model
899
+ language_model_output = self.language_model(
900
+ input_ids=None,
901
+ attention_mask=multimodal_attention_mask,
902
+ position_ids=None,
903
+ past_key_values=None,
904
+ inputs_embeds=multimodal_embeddings,
905
+ labels=None,
906
+ use_cache=None,
907
+ output_attentions=False,
908
+ output_hidden_states=True,
909
+ return_dict=True,
910
+ )
911
+
912
+ # Extract hidden states for action tokens
913
+ last_hidden_states = language_model_output.hidden_states[-1] # (B, seq_len, D)
914
+ actions_hidden_states = last_hidden_states[
915
+ :,
916
+ NUM_PATCHES + NUM_PROMPT_TOKENS : NUM_PATCHES + NUM_PROMPT_TOKENS + ACTION_DIM * NUM_ACTIONS_CHUNK,
917
+ :,
918
+ ] # (B, act_chunk_len, D)
919
+
920
+ # Handle different prediction methods
921
+ if action_head is not None:
922
+ # L1 regression prediction
923
+ normalized_actions = action_head.predict_action(actions_hidden_states)
924
+ normalized_actions = normalized_actions.reshape(NUM_ACTIONS_CHUNK, ACTION_DIM)
925
+ normalized_actions = normalized_actions.float().cpu().detach().numpy()
926
+ else:
927
+ # Discrete token-based prediction
928
+ predicted_action_token_ids = (
929
+ language_model_output.logits[
930
+ :,
931
+ NUM_PATCHES + NUM_PROMPT_TOKENS : NUM_PATCHES + NUM_PROMPT_TOKENS + ACTION_DIM * NUM_ACTIONS_CHUNK,
932
+ ]
933
+ .argmax(dim=2)
934
+ .cpu()
935
+ .numpy()
936
+ )
937
+ discretized_actions = self.vocab_size - predicted_action_token_ids
938
+ discretized_actions = np.clip(discretized_actions - 1, a_min=0, a_max=self.bin_centers.shape[0] - 1)
939
+ normalized_actions = self.bin_centers[discretized_actions]
940
+ normalized_actions = normalized_actions.reshape(NUM_ACTIONS_CHUNK, ACTION_DIM)
941
+
942
+ return normalized_actions, actions_hidden_states
943
+
944
+ def predict_action(
945
+ self,
946
+ input_ids: Optional[torch.LongTensor] = None,
947
+ unnorm_key: Optional[str] = None,
948
+ proprio=None,
949
+ proprio_projector=None,
950
+ action_head=None,
951
+ noisy_action_projector=None,
952
+ use_film: bool = False,
953
+ **kwargs: str,
954
+ ) -> np.ndarray:
955
+ """Predict actions from input sequence, with options for different prediction methods.
956
+
957
+ Args:
958
+ input_ids: Input token ids
959
+ unnorm_key: Key for unnormalization statistics
960
+ proprio: Proprioceptive features
961
+ proprio_projector: Projector for proprioceptive features
962
+ action_head: Optional head for L1 regression or diffusion-based prediction
963
+ noisy_action_projector: Projector for noisy actions in diffusion-based prediction
964
+ use_film: Whether to use FiLM conditioning
965
+ **kwargs: Additional arguments including pixel_values and attention_mask
966
+
967
+ Returns:
968
+ Tuple of (unnormalized_actions, action_hidden_states)
969
+ """
970
+ # If the special empty token ('') does not already appear after the colon (':') token in the prompt
971
+ # (after "OUT:" or "ASSISTANT:"), insert it to match the inputs seen at training time
972
+ if not torch.all(input_ids[:, -1] == 29871):
973
+ input_ids = torch.cat(
974
+ (input_ids, torch.unsqueeze(torch.Tensor([29871]).long(), dim=0).to(input_ids.device)), dim=1
975
+ )
976
+
977
+ pixel_values = kwargs["pixel_values"]
978
+ attention_mask = kwargs["attention_mask"]
979
+
980
+ # Create fake labels tensor (needed for action mask)
981
+ labels = input_ids.clone()
982
+ labels[:] = IGNORE_INDEX
983
+
984
+ # Get number of tokens in prompt (excluding the start token)
985
+ NUM_PROMPT_TOKENS = input_ids.shape[-1] - 1 # Subtract action tokens and stop token
986
+
987
+ # Prepare inputs by adding necessary tokens
988
+ input_ids, attention_mask = self._prepare_input_for_action_prediction(input_ids, attention_mask)
989
+
990
+ # Update labels tensor for action mask computation later
991
+ labels = self._prepare_labels_for_action_prediction(labels, input_ids)
992
+
993
+ # Get input embeddings and action masks
994
+ input_embeddings = self.get_input_embeddings()(input_ids)
995
+ all_actions_mask = self._process_action_masks(labels)
996
+
997
+ # Extract language embeddings
998
+ language_embeddings = input_embeddings[~all_actions_mask].reshape(
999
+ input_embeddings.shape[0], -1, input_embeddings.shape[2]
1000
+ )
1001
+
1002
+ # Process vision features
1003
+ projected_patch_embeddings = self._process_vision_features(pixel_values, language_embeddings, use_film)
1004
+
1005
+ # Add proprioceptive features if provided
1006
+ use_proprio = proprio_projector is not None and proprio is not None
1007
+ if use_proprio:
1008
+ proprio = torch.Tensor(proprio).to(projected_patch_embeddings.device, dtype=projected_patch_embeddings.dtype)
1009
+ projected_patch_embeddings = self._process_proprio_features(
1010
+ projected_patch_embeddings, proprio, proprio_projector
1011
+ )
1012
+
1013
+ # Use diffusion if provided, otherwise use regression or discrete prediction
1014
+ use_diffusion = noisy_action_projector is not None and hasattr(action_head, "noise_scheduler")
1015
+
1016
+ # Calculate number of patches (including proprio token and/or diffusion timestep embedding if present)
1017
+ NUM_PATCHES = self.vision_backbone.get_num_patches() * self.vision_backbone.get_num_images_in_input()
1018
+ if use_proprio:
1019
+ NUM_PATCHES += 1
1020
+ if use_diffusion:
1021
+ NUM_PATCHES += 1
1022
+
1023
+ if use_diffusion:
1024
+ # Sample random noise with shape equal to output action, used as the starting state for reverse diffusion
1025
+ noise = torch.randn(
1026
+ size=(1, NUM_ACTIONS_CHUNK, ACTION_DIM), device=input_embeddings.device, dtype=input_embeddings.dtype
1027
+ )
1028
+
1029
+ # Run diffusion-based prediction
1030
+ normalized_actions, actions_hidden_states = self._run_diffusion_prediction(
1031
+ input_embeddings,
1032
+ all_actions_mask,
1033
+ noise,
1034
+ action_head,
1035
+ projected_patch_embeddings,
1036
+ labels,
1037
+ attention_mask,
1038
+ NUM_PATCHES,
1039
+ NUM_PROMPT_TOKENS,
1040
+ noisy_action_projector,
1041
+ )
1042
+ else:
1043
+ # Run regression or discrete token-based prediction
1044
+ normalized_actions, actions_hidden_states = self._regression_or_discrete_prediction(
1045
+ input_embeddings,
1046
+ all_actions_mask,
1047
+ projected_patch_embeddings,
1048
+ attention_mask,
1049
+ labels,
1050
+ NUM_PATCHES,
1051
+ NUM_PROMPT_TOKENS,
1052
+ action_head,
1053
+ )
1054
+
1055
+ # Unnormalize predicted actions
1056
+ actions = self._unnormalize_actions(normalized_actions, unnorm_key)
1057
+
1058
+ return actions, actions_hidden_states
1059
+
1060
+ @staticmethod
1061
+ def _check_unnorm_key(norm_stats: Dict[str, Dict[str, Any]], unnorm_key: Optional[str]) -> str:
1062
+ """Validate and resolve the unnormalization key for action statistics"""
1063
+ if unnorm_key is None:
1064
+ assert len(norm_stats) == 1, (
1065
+ f"Your model was trained on more than one dataset, "
1066
+ f"please pass a `unnorm_key` from the following options to choose the statistics "
1067
+ f"used for un-normalizing actions: {norm_stats.keys()}"
1068
+ )
1069
+ unnorm_key = next(iter(norm_stats.keys()))
1070
+
1071
+ assert unnorm_key in norm_stats, (
1072
+ f"The `unnorm_key` you chose is not in the set of available dataset statistics, "
1073
+ f"please choose from: {norm_stats.keys()}"
1074
+ )
1075
+ return unnorm_key
1076
+
1077
+ def get_action_dim(self, unnorm_key: Optional[str] = None) -> int:
1078
+ """Get the dimensionality of the policy's action space."""
1079
+ unnorm_key = self._check_unnorm_key(self.norm_stats, unnorm_key)
1080
+ return len(self.norm_stats[unnorm_key]["action"]["min"])
1081
+
1082
+ def get_action_stats(self, unnorm_key: Optional[str] = None) -> Dict[str, Any]:
1083
+ """Get all the logged statistics for the given dataset."""
1084
+ unnorm_key = self._check_unnorm_key(self.norm_stats, unnorm_key)
1085
+ return self.norm_stats[unnorm_key]["action"]
preprocessor_config.json ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "auto_map": {
3
+ "AutoImageProcessor": "processing_prismatic.PrismaticImageProcessor",
4
+ "AutoProcessor": "processing_prismatic.PrismaticProcessor"
5
+ },
6
+ "image_processor_type": "PrismaticImageProcessor",
7
+ "image_resize_strategy": "resize-naive",
8
+ "input_sizes": [
9
+ [
10
+ 3,
11
+ 224,
12
+ 224
13
+ ],
14
+ [
15
+ 3,
16
+ 224,
17
+ 224
18
+ ]
19
+ ],
20
+ "interpolations": [
21
+ "bicubic",
22
+ "bicubic"
23
+ ],
24
+ "means": [
25
+ [
26
+ 0.485,
27
+ 0.456,
28
+ 0.406
29
+ ],
30
+ [
31
+ 0.5,
32
+ 0.5,
33
+ 0.5
34
+ ]
35
+ ],
36
+ "processor_class": "PrismaticProcessor",
37
+ "stds": [
38
+ [
39
+ 0.229,
40
+ 0.224,
41
+ 0.225
42
+ ],
43
+ [
44
+ 0.5,
45
+ 0.5,
46
+ 0.5
47
+ ]
48
+ ],
49
+ "tvf_crop_params": [
50
+ {
51
+ "output_size": [
52
+ 224,
53
+ 224
54
+ ]
55
+ },
56
+ {
57
+ "output_size": [
58
+ 224,
59
+ 224
60
+ ]
61
+ }
62
+ ],
63
+ "tvf_do_letterbox": false,
64
+ "tvf_letterbox_fill": null,
65
+ "tvf_normalize_params": [
66
+ {
67
+ "inplace": false,
68
+ "mean": [
69
+ 0.484375,
70
+ 0.455078125,
71
+ 0.40625
72
+ ],
73
+ "std": [
74
+ 0.228515625,
75
+ 0.2236328125,
76
+ 0.224609375
77
+ ]
78
+ },
79
+ {
80
+ "inplace": false,
81
+ "mean": [
82
+ 0.5,
83
+ 0.5,
84
+ 0.5
85
+ ],
86
+ "std": [
87
+ 0.5,
88
+ 0.5,
89
+ 0.5
90
+ ]
91
+ }
92
+ ],
93
+ "tvf_resize_params": [
94
+ {
95
+ "antialias": true,
96
+ "interpolation": 3,
97
+ "max_size": null,
98
+ "size": [
99
+ 224,
100
+ 224
101
+ ]
102
+ },
103
+ {
104
+ "antialias": true,
105
+ "interpolation": 3,
106
+ "max_size": null,
107
+ "size": [
108
+ 224,
109
+ 224
110
+ ]
111
+ }
112
+ ],
113
+ "use_fused_vision_backbone": true
114
+ }
processing_prismatic.py ADDED
@@ -0,0 +1,257 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ processing_prismatic.py
3
+
4
+ HuggingFace-style preprocessor definitions for Prismatic VLMs, inheriting from `ProcessorMixin`. Default configuration
5
+ specifies `siglip-224px+7b`.
6
+ """
7
+
8
+ from typing import Any, ClassVar, List, Optional, Tuple, Union
9
+
10
+ import timm.data
11
+ import torch
12
+ import torchvision.transforms.functional as TVF
13
+ from PIL import Image
14
+ from torchvision.transforms import CenterCrop, Compose, Normalize, Resize, ToTensor
15
+ from transformers import PreTrainedTokenizerBase
16
+ from transformers.image_processing_utils import BatchFeature, ImageProcessingMixin
17
+ from transformers.processing_utils import ProcessorMixin
18
+ from transformers.tokenization_utils import PaddingStrategy, PreTokenizedInput, TextInput, TruncationStrategy
19
+ from transformers.utils import TensorType
20
+
21
+
22
+ # === Image Processing ===
23
+ def letterbox_pad_transform(image: Image.Image, padding_fill_value: Tuple[int, int, int]) -> Image.Image:
24
+ """Given a PIL.Image, pad to square by adding a symmetric border around the height/width."""
25
+ (w, h), max_wh = image.size, max(image.size)
26
+ horizontal_pad, vertical_pad = int((max_wh - w) / 2), int((max_wh - h) / 2)
27
+ padding = (horizontal_pad, vertical_pad, horizontal_pad, vertical_pad)
28
+
29
+ return TVF.pad(image, padding, fill=padding_fill_value, padding_mode="constant")
30
+
31
+
32
+ class PrismaticImageProcessor(ImageProcessingMixin):
33
+ model_input_names: ClassVar[List[str]] = ["pixel_values"]
34
+
35
+ def __init__(
36
+ self,
37
+ use_fused_vision_backbone: bool = False,
38
+ image_resize_strategy: str = "letterbox",
39
+ input_sizes: Optional[List[Tuple[int, int, int]]] = None,
40
+ interpolations: Optional[List[str]] = None,
41
+ means: Optional[List[Tuple[float, float, float]]] = None,
42
+ stds: Optional[List[Tuple[float, float, float]]] = None,
43
+ **kwargs: str,
44
+ ) -> None:
45
+ """
46
+ Initialize a PrismaticImageProcessor as a wrapper around a torchvision transform; this transform will be
47
+ created by TIMM, and edited to follow our custom `image_resize_strategy` logic.
48
+
49
+ @param use_fused_vision_backbone: Boolean indicating single or fused (dual) vision backbone
50
+ @param image_resize_strategy: Prismatic image resize strategy in < resize-naive | resize-crop | letterbox >
51
+ @param input_size: [TIMM :: `data_cfg`] Input image size as tuple (channels, width, height)
52
+ @param interpolation: [TIMM :: `data_cfg`] Interpolation as string (default: "bicubic")
53
+ @param mean: [TIMM :: `data_cfg`] Normalization mean as float tuple (or two-tuple if `fused_backbone`)
54
+ @param std: [TIMM :: `data_cfg`] Normalization std as float tuple (or two-tuple if `fused_backbone`)
55
+ """
56
+ self.use_fused_vision_backbone = use_fused_vision_backbone
57
+ self.image_resize_strategy = image_resize_strategy
58
+
59
+ # Handle `None` default values
60
+ input_sizes = [(3, 224, 224)] if input_sizes is None else input_sizes
61
+ means = [(0.5, 0.5, 0.5)] if means is None else means
62
+ stds = [(0.5, 0.5, 0.5)] if stds is None else stds
63
+
64
+ # TIMM `data_cfg` Parameters
65
+ self.input_sizes, self.interpolations, self.means, self.stds = input_sizes, interpolations, means, stds
66
+
67
+ # Grab torchvision transforms via TIMM =>> need to parse for specific "functional" transform values!
68
+ self.tvf_resize_params, self.tvf_crop_params, self.tvf_normalize_params = [], [], []
69
+ self.tvf_do_letterbox, self.tvf_letterbox_fill = False, None
70
+
71
+ for idx in range(len(input_sizes)):
72
+ transform = timm.data.create_transform(
73
+ input_size=self.input_sizes[idx],
74
+ interpolation=self.interpolations[idx],
75
+ mean=self.means[idx],
76
+ std=self.stds[idx],
77
+ crop_pct=1.0, # Set to 1.0 to ignore cropping (initial Resize sets `input_size`)
78
+ crop_mode="center", # Default crop mode -- no-op when `crop_pct == 1.0`
79
+ is_training=False, # No image augmentations when loading the transform!
80
+ )
81
+
82
+ # [Validation] Ensure appropriate transform structure, expected sizes
83
+ if not (
84
+ isinstance(transform, Compose)
85
+ and (len(transform.transforms) == 4)
86
+ and isinstance(transform.transforms[0], Resize)
87
+ and isinstance(transform.transforms[1], CenterCrop)
88
+ and isinstance(transform.transforms[2], ToTensor)
89
+ and isinstance(transform.transforms[3], Normalize)
90
+ and (transform.transforms[0].size == self.input_sizes[idx][-1])
91
+ and (transform.transforms[1].size == self.input_sizes[idx][-2:])
92
+ ):
93
+ raise ValueError(f"Unexpected TIMM image transformation structure/sizes: `{transform}`")
94
+
95
+ # HF Image Processors *must* be JSON-serializable; as such, cannot have torchvision. as an attribute.
96
+ # => Instead, we're going to parse the transform and call "torchvision.transforms.functional" (`tvf`)
97
+ resize_t, crop_t, norm_t = transform.transforms[0], transform.transforms[1], transform.transforms[3]
98
+ self.tvf_resize_params.append(
99
+ {
100
+ "size": resize_t.size,
101
+ "interpolation": TVF.pil_modes_mapping[resize_t.interpolation],
102
+ "max_size": None,
103
+ "antialias": True,
104
+ }
105
+ )
106
+ self.tvf_crop_params.append({"output_size": crop_t.size})
107
+ self.tvf_normalize_params.append(
108
+ {
109
+ "mean": norm_t.mean.float().numpy().tolist(),
110
+ "std": norm_t.std.float().numpy().tolist(),
111
+ "inplace": False,
112
+ }
113
+ )
114
+ self.tvf_do_letterbox, self.tvf_letterbox_fill = False, None
115
+
116
+ # Handle Prismatic `image_resize_strategy`
117
+ if self.image_resize_strategy == "resize-naive":
118
+ self.tvf_resize_params[idx]["size"] = (resize_t.size, resize_t.size)
119
+ elif self.image_resize_strategy == "letterbox":
120
+ self.tvf_do_letterbox, self.tvf_letterbox_fill = True, tuple([int(x * 255) for x in self.means[idx]])
121
+ elif self.image_resize_strategy == "resize-crop":
122
+ pass
123
+ else:
124
+ raise ValueError(f"Image resize strategy `{self.image_resize_strategy}` is not supported!")
125
+
126
+ # Dispatch **kwargs to super()
127
+ super().__init__(**kwargs)
128
+
129
+ def apply_transform(self, img: Image.Image) -> torch.Tensor:
130
+ """Apply `functional` variant of TIMM's Transform = Compose([Resize -> CenterCrop -> ToTensor -> Normalize])"""
131
+ if self.tvf_do_letterbox:
132
+ img = letterbox_pad_transform(img, self.tvf_letterbox_fill)
133
+
134
+ # [Contract] Fused Backbones expect "channel-stacked" inputs; we'll unpack on the model side!
135
+ imgs_t = []
136
+ for idx in range(len(self.input_sizes)):
137
+ img_idx = TVF.resize(img, **self.tvf_resize_params[idx])
138
+ img_idx = TVF.center_crop(img_idx, **self.tvf_crop_params[idx])
139
+ img_idx_t = TVF.to_tensor(img_idx)
140
+ img_idx_t = TVF.normalize(img_idx_t, **self.tvf_normalize_params[idx])
141
+ imgs_t.append(img_idx_t)
142
+
143
+ # [Contract] `imgs_t` is a list of Tensors of shape [3, input_size, input_size]; stack along dim = 0
144
+ img_t = torch.vstack(imgs_t)
145
+
146
+ return img_t
147
+
148
+ def preprocess(
149
+ self,
150
+ images: Union[Image.Image, List[Image.Image]],
151
+ return_tensors: Optional[Union[str, TensorType]] = None,
152
+ **_: str,
153
+ ) -> BatchFeature:
154
+ """
155
+ Preprocess an image (or batch of images); note that unlike the `transformers :: BaseImageProcessor` we
156
+ explicitly only handle PIL.Image.Image instances for simplicity.
157
+
158
+ @param images: A (batch of) PIL.Image.Image instance(s) to preprocess.
159
+ @param return_tensors: BatchFeature default Tensor format (e.g., "pt" for torch); if None, returns np.ndarray
160
+
161
+ @return: Instance of `transformers :: BatchFeature` with a single key "pixel_values"
162
+ """
163
+ if not isinstance(images, list):
164
+ images = [images]
165
+
166
+ # Apply `self.img_transform` to each image (will return list of torch.Tensors); stack into "batched" Tensor
167
+ pixel_values = torch.stack([self.apply_transform(img.convert("RGB")) for img in images])
168
+
169
+ # Return BatchFeature =>> note that for compatibility, constructor expects Dict[str, np.ndarray], so we convert
170
+ return BatchFeature(data={"pixel_values": pixel_values.float().numpy()}, tensor_type=return_tensors)
171
+
172
+ def __call__(self, images: Union[Image.Image, List[Image.Image]], **kwargs) -> BatchFeature:
173
+ return self.preprocess(images, **kwargs)
174
+
175
+
176
+ # === PrismaticProcessor =>> Wraps both ImageProcessor and Tokenizer ===
177
+ # =>> https://github.com/huggingface/transformers/blob/main/src/transformers/models/llava/processing_llava.py
178
+ class PrismaticProcessor(ProcessorMixin):
179
+ attributes: ClassVar[List[str]] = ["image_processor", "tokenizer"]
180
+ image_processor_class: str = "AutoImageProcessor"
181
+ tokenizer_class: str = "AutoTokenizer"
182
+
183
+ def __init__(
184
+ self,
185
+ image_processor: Optional[ImageProcessingMixin] = None,
186
+ tokenizer: Optional[PreTrainedTokenizerBase] = None,
187
+ ) -> None:
188
+ super().__init__(image_processor, tokenizer)
189
+
190
+ def __call__(
191
+ self,
192
+ text: Union[TextInput, PreTokenizedInput, List[TextInput], List[PreTokenizedInput]],
193
+ images: Union[Image.Image, List[Image.Image]],
194
+ padding: Union[bool, str, PaddingStrategy] = False,
195
+ truncation: Optional[Union[bool, str, TruncationStrategy]] = None,
196
+ max_length: Optional[int] = None,
197
+ return_tensors: Optional[Union[str, TensorType]] = TensorType.PYTORCH,
198
+ ) -> BatchFeature:
199
+ """
200
+ Preprocess a given (batch) of text/images for a Prismatic VLM; forwards text to the underlying LLM's tokenizer,
201
+ forwards images to PrismaticImageProcessor.
202
+
203
+ @param text: The (batch) of text to encode; must be a string or list of strings.
204
+ @param images: A (batch of) PIL.Image.Image instance(s) to preprocess.
205
+ @param padding: Sequence padding strategy (if multiple specified) in < True = "longest" | "max_length" | False >
206
+ @param truncation: Truncation strategy for the output sequences; requires `max_length` to be specified
207
+ @param max_length: Maximum length (in tokens) to truncate
208
+ @param return_tensors: Type of return tensors (usually "pt" or TensorType.PYTORCH)
209
+
210
+ @return: BatchFeature with keys for `input_ids`, `attention_mask` and `pixel_values`.
211
+ """
212
+ pixel_values = self.image_processor(images, return_tensors=return_tensors)["pixel_values"]
213
+ text_inputs = self.tokenizer(
214
+ text, return_tensors=return_tensors, padding=padding, truncation=truncation, max_length=max_length
215
+ )
216
+
217
+ # [Validate] Need same number of images and text inputs!
218
+ if pixel_values.shape[0] != text_inputs.input_ids.shape[0]:
219
+ raise ValueError("Batch is malformed; expected same number of images and text inputs!")
220
+
221
+ return BatchFeature(data={**text_inputs, "pixel_values": pixel_values})
222
+
223
+ # === Tokenizer Dispatch Utilities =>> check `PreTrainedTokenizerBase` for documentation ===
224
+ def batch_decode(
225
+ self,
226
+ sequences: Union[List[int], List[List[int]], torch.Tensor, Any], # `Any` = np.ndarray | tf.Tensor
227
+ skip_special_tokens: bool = False,
228
+ clean_up_tokenization_spaces: Optional[bool] = None,
229
+ **kwargs: str,
230
+ ) -> List[str]:
231
+ return self.tokenizer.batch_decode(
232
+ sequences=sequences,
233
+ skip_special_tokens=skip_special_tokens,
234
+ clean_up_tokenization_spaces=clean_up_tokenization_spaces,
235
+ **kwargs,
236
+ )
237
+
238
+ def decode(
239
+ self,
240
+ token_ids: Union[int, List[int], torch.Tensor, Any], # `Any` = np.ndarray | tf.Tensor
241
+ skip_special_tokens: bool = False,
242
+ clean_up_tokenization_spaces: Optional[bool] = None,
243
+ **kwargs: str,
244
+ ) -> str:
245
+ return self.tokenizer.decode(
246
+ token_ids=token_ids,
247
+ skip_special_tokens=skip_special_tokens,
248
+ clean_up_tokenization_spaces=clean_up_tokenization_spaces,
249
+ **kwargs,
250
+ )
251
+
252
+ @property
253
+ def model_input_names(self) -> List[str]:
254
+ tokenizer_input_names = self.tokenizer.model_input_names
255
+ image_processor_input_names = self.image_processor.model_input_names
256
+
257
+ return list(dict.fromkeys(tokenizer_input_names + image_processor_input_names))
processor_config.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "auto_map": {
3
+ "AutoProcessor": "processing_prismatic.PrismaticProcessor"
4
+ },
5
+ "processor_class": "PrismaticProcessor"
6
+ }
proprio_projector--150000_checkpoint.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:491e30d2aaa1d9ff33521a93a511b1828f8deb33515dd2b43b470ade74569221
3
+ size 67275256
special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "</s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "<PAD>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "unk_token": {
24
+ "content": "<unk>",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
3
+ size 499723
tokenizer_config.json ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "added_tokens_decoder": {
5
+ "0": {
6
+ "content": "<unk>",
7
+ "lstrip": false,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false,
11
+ "special": true
12
+ },
13
+ "1": {
14
+ "content": "<s>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false,
19
+ "special": true
20
+ },
21
+ "2": {
22
+ "content": "</s>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": false,
26
+ "single_word": false,
27
+ "special": true
28
+ },
29
+ "32000": {
30
+ "content": "<PAD>",
31
+ "lstrip": false,
32
+ "normalized": false,
33
+ "rstrip": false,
34
+ "single_word": false,
35
+ "special": true
36
+ }
37
+ },
38
+ "auto_map": {
39
+ "AutoProcessor": "processing_prismatic.PrismaticProcessor"
40
+ },
41
+ "bos_token": "<s>",
42
+ "clean_up_tokenization_spaces": false,
43
+ "eos_token": "</s>",
44
+ "legacy": false,
45
+ "model_max_length": 2048,
46
+ "pad_token": "<PAD>",
47
+ "padding_side": "right",
48
+ "processor_class": "PrismaticProcessor",
49
+ "sp_model_kwargs": {},
50
+ "tokenizer_class": "LlamaTokenizer",
51
+ "unk_token": "<unk>",
52
+ "use_default_system_prompt": false
53
+ }