Aedelon commited on
Commit
1428b16
·
verified ·
1 Parent(s): bfbe01e

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +140 -0
README.md ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-sa-4.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - mast3r
7
+ - dune
8
+ - dust3r
9
+ - 3d-vision
10
+ - stereo-matching
11
+ - depth-estimation
12
+ - point-cloud
13
+ - sfm
14
+ - slam
15
+ - robotics
16
+ pipeline_tag: depth-estimation
17
+ library_name: mast3r-runtime
18
+ ---
19
+
20
+ # MASt3R & DUNE Checkpoints (SafeTensors)
21
+
22
+ Pre-trained checkpoints for **MASt3R** (Matching And Stereo 3D Reconstruction) and **DUNE** (Dense UNconstrained Estimation) models, converted to SafeTensors format for efficient C++/embedded inference.
23
+
24
+ ## Models
25
+
26
+ | Model | Resolution | Encoder | Size | Use Case |
27
+ |-------|------------|---------|------|----------|
28
+ | `dune_vit_small_336` | 336x336 | ViT-S/14 | ~1.3 GB | Real-time drone/embedded |
29
+ | `dune_vit_small_448` | 448x448 | ViT-S/14 | ~1.3 GB | Fast inference |
30
+ | `dune_vit_base_336` | 336x336 | ViT-B/14 | ~1.7 GB | Balanced speed/quality |
31
+ | `dune_vit_base_448` | 448x448 | ViT-B/14 | ~1.7 GB | High quality |
32
+ | `mast3r_vit_large` | 512x512 | ViT-L/14 | ~2.9 GB | Maximum precision |
33
+
34
+ ## Architecture
35
+
36
+ - **Encoder**: DINOv2-based Vision Transformer (DUNE-trained)
37
+ - **Decoder**: MASt3R decoder with CatMLP+DPT heads
38
+ - **Outputs**: Dense 3D points + descriptors for matching
39
+
40
+ ```
41
+ Image Pair → DUNE Encoder → MASt3R Decoder → 3D Points + Descriptors
42
+ ```
43
+
44
+ ## Usage
45
+
46
+ ### With mast3r-runtime (recommended)
47
+
48
+ ```bash
49
+ pip install mast3r-runtime
50
+
51
+ # Download and convert
52
+ mast3r-runtime download dune_vit_small_336
53
+ mast3r-runtime convert dune_vit_small_336 --dtype fp16
54
+ ```
55
+
56
+ ### Direct download
57
+
58
+ ```python
59
+ from huggingface_hub import hf_hub_download
60
+
61
+ # Download encoder
62
+ encoder = hf_hub_download(
63
+ repo_id="aedelon/mast3r-checkpoints",
64
+ filename="dune_vit_small_336/encoder.safetensors"
65
+ )
66
+
67
+ # Download decoder
68
+ decoder = hf_hub_download(
69
+ repo_id="aedelon/mast3r-checkpoints",
70
+ filename="dune_vit_small_336/decoder.safetensors"
71
+ )
72
+ ```
73
+
74
+ ## Credits & Acknowledgments
75
+
76
+ These models are converted from the original checkpoints released by **Naver Labs Europe**.
77
+
78
+ ### MASt3R
79
+ > **Grounding Image Matching in 3D with MASt3R**
80
+ > Vincent Leroy, Yohann Cabon, Jérôme Revaud
81
+ > arXiv 2024
82
+
83
+ ```bibtex
84
+ @article{leroy2024mast3r,
85
+ title={Grounding Image Matching in 3D with MASt3R},
86
+ author={Leroy, Vincent and Cabon, Yohann and Revaud, J{\'e}r{\^o}me},
87
+ journal={arXiv preprint arXiv:2406.09756},
88
+ year={2024}
89
+ }
90
+ ```
91
+
92
+ ### DUNE
93
+ > **DUNE: Dense UNconstrained Estimation for 3D Vision**
94
+ > Vincent Leroy, Yohann Cabon, Jérôme Revaud
95
+ > CVPR 2025
96
+
97
+ ```bibtex
98
+ @inproceedings{leroy2025dune,
99
+ title={DUNE: Dense UNconstrained Estimation for 3D Vision},
100
+ author={Leroy, Vincent and Cabon, Yohann and Revaud, J{\'e}r{\^o}me},
101
+ booktitle={CVPR},
102
+ year={2025}
103
+ }
104
+ ```
105
+
106
+ ### DUSt3R
107
+ > **DUSt3R: Geometric 3D Vision Made Easy**
108
+ > Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, Jérôme Revaud
109
+ > CVPR 2024
110
+
111
+ ```bibtex
112
+ @inproceedings{wang2024dust3r,
113
+ title={DUSt3R: Geometric 3D Vision Made Easy},
114
+ author={Wang, Shuzhe and Leroy, Vincent and Cabon, Yohann and Chidlovskii, Boris and Revaud, J{\'e}r{\^o}me},
115
+ booktitle={CVPR},
116
+ year={2024}
117
+ }
118
+ ```
119
+
120
+ ## Original Repositories
121
+
122
+ - [naver/mast3r](https://github.com/naver/mast3r) - MASt3R official implementation
123
+ - [naver/dune](https://github.com/naver/dune) - DUNE official implementation
124
+ - [naver/dust3r](https://github.com/naver/dust3r) - DUSt3R official implementation
125
+
126
+ ## License
127
+
128
+ The model weights are released under **CC BY-NC-SA 4.0** (Creative Commons Attribution-NonCommercial-ShareAlike 4.0).
129
+
130
+ - **Attribution**: Credit Naver Labs Europe
131
+ - **NonCommercial**: No commercial use without permission
132
+ - **ShareAlike**: Derivatives must use same license
133
+
134
+ For commercial licensing, contact [Naver Labs Europe](https://europe.naverlabs.com/).
135
+
136
+ ## Converted by
137
+
138
+ [Delanoe Pirard / Aedelon](https://github.com/aedelon) - [mast3r-runtime](https://github.com/aedelon/mast3r-runtime)
139
+
140
+ SafeTensors conversion for embedded/C++ inference (Apache 2.0 for runtime code).