fernandotonon commited on
Commit
284a074
·
verified ·
1 Parent(s): 16643b0

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +64 -0
README.md ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - onnx
5
+ - image-to-3d
6
+ - 3d-reconstruction
7
+ - triplane
8
+ - qtmesheditor
9
+ library_name: onnx
10
+ pipeline_tag: image-to-3d
11
+ ---
12
+
13
+ # TripoSR — ONNX export
14
+
15
+ **ONNX re-export of [stabilityai/TripoSR](https://huggingface.co/stabilityai/TripoSR)**
16
+ (Tripo AI + Stability AI — MIT code + MIT weights): single-image → 3D
17
+ reconstruction via a triplane radiance field. All credit for the original
18
+ weights goes to Tripo AI and Stability AI.
19
+
20
+ Exported for **[QtMeshEditor](https://github.com/fernandotonon/QtMeshEditor)**
21
+ (epic #764), powering `qtmesh generate3d`, the Inspector's **AI: Image → 3D**
22
+ section, and the `generate_mesh_from_image` MCP tool — local inference via
23
+ ONNX Runtime, with surface extraction done by a native marching-cubes
24
+ implementation (upstream's `torchmcubes` is torch/GPU-only).
25
+
26
+ > The files QtMeshEditor downloads at runtime live in the shared
27
+ > [`fernandotonon/QtMeshEditor-models`](https://huggingface.co/fernandotonon/QtMeshEditor-models)
28
+ > repo under `triposr/`. This repo is the standalone model card + mirror.
29
+
30
+ ## Files
31
+
32
+ | file | role |
33
+ |---|---|
34
+ | `triposr_encoder.onnx` | fp32 encoder (best quality, ~1.7 GB): `image [1,3,512,512]` → `scene_codes [1,3,40,64,64]` (triplane) |
35
+ | `triposr_encoder_int8.onnx` | int8 encoder tier (~430 MB, slight quality loss) |
36
+ | `triposr_decoder.onnx` | per-point field decoder: `scene_codes` + `points [1,P,3]` → `density [1,P,1]`, `color [1,P,3]` |
37
+
38
+ An fp16 tier is deliberately absent: TripoSR's attention has a hardcoded
39
+ Cast-to-float32 that the ONNX fp16 converters cannot rewrite.
40
+
41
+ ## Inference contract
42
+
43
+ - Input image: 512×512 RGB in `[0,1]`, subject isolated (upstream composites
44
+ over gray-128 and re-pads the foreground to a 0.85 ratio; QtMeshEditor uses
45
+ U²-Net for background removal first).
46
+ - Query the decoder in **chunks** — generating the full `res³` grid up front
47
+ OOMs at high resolutions.
48
+ - Surface = marching cubes on `density − threshold` at iso 0, threshold 25.0,
49
+ within radius 0.87. The field is inside-positive; emit flipped winding
50
+ (`v0,v2,v1`) to keep faces outward.
51
+ - Output is not +Y-up: bake −90°X then +90°Y to stand the model upright
52
+ facing forward.
53
+ - The decoder's colour output can also be used as an image-conditioned colour
54
+ field for texture baking (per-texel queries after a UV unwrap).
55
+
56
+ ## Reproducing
57
+
58
+ `scripts/export-triposr-onnx.py` in the QtMeshEditor repo (one-time, offline;
59
+ `transformers==4.35.0`, `torchmcubes` stubbed, frozen ViT positional
60
+ encoding; emits the int8 variant unless `--no-quant`).
61
+
62
+ ## License
63
+
64
+ MIT (same as the upstream code and weights). Credit: Tripo AI + Stability AI.