fernandotonon commited on
Commit
be167ee
·
verified ·
1 Parent(s): 290587b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +72 -0
README.md ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - onnx
5
+ - image-to-3d
6
+ - 3d-generation
7
+ - rectified-flow
8
+ - diffusion-transformer
9
+ - qtmesheditor
10
+ library_name: onnx
11
+ pipeline_tag: image-to-3d
12
+ ---
13
+
14
+ # TripoSG — ONNX export (staged graphs)
15
+
16
+ **ONNX re-export of [VAST-AI/TripoSG](https://huggingface.co/VAST-AI/TripoSG)**
17
+ (SIGGRAPH 2025 — MIT code + MIT weights): a 1.5B-parameter rectified-flow
18
+ diffusion transformer over an SDF VAE, single image → high-fidelity 3D
19
+ geometry (≈ commercial Tripo 2.0 quality). All credit for the original
20
+ weights goes to VAST-AI-Research.
21
+
22
+ Exported as **four staged graphs** so the flow loop runs in plain host code —
23
+ for **[QtMeshEditor](https://github.com/fernandotonon/QtMeshEditor)**
24
+ (`qtmesh generate3d --backend triposg`, GUI Backend dropdown, MCP `backend`
25
+ arg), local inference via ONNX Runtime + native marching cubes.
26
+
27
+ > The files QtMeshEditor downloads at runtime live in the shared
28
+ > [`fernandotonon/QtMeshEditor-models`](https://huggingface.co/fernandotonon/QtMeshEditor-models)
29
+ > repo under `triposg/`. This repo is the standalone model card + mirror.
30
+
31
+ ## Files
32
+
33
+ | file | role |
34
+ |---|---|
35
+ | `triposg_image_encoder.onnx` | DINOv2-224 image encoder (mean/std baked in; CFG unconditional = zeros) |
36
+ | `triposg_dit_step.onnx` + `.data` | one DiT flow step (fp32, ~5.8 GB external weights) |
37
+ | `triposg_vae_latents.onnx` | VAE latent KV-cache graph — run **once** per generation |
38
+ | `triposg_vae_decoder.onnx` | per-point SDF field decoder |
39
+
40
+ An int8 DiT tier exists in the aggregate repo but is **not recommended**: even
41
+ per-channel-quantized, the 1.5B DiT degrades to blobs over the 25-step CFG
42
+ flow loop, and dynamic-int8 MatMuls are no faster than fp32 on ARM.
43
+
44
+ ## Inference contract
45
+
46
+ - **Flow loop (host code):** `σᵢ = 1 − i/N`, timestep `1000·σᵢ`, update
47
+ `x += (σᵢ − σᵢ₊₁)·v` — note the sign is **opposite** of stock diffusers
48
+ `FlowMatchEulerDiscreteScheduler`. CFG as two batch-1 calls, guidance 7.0,
49
+ 25 steps default.
50
+ - Background removal should composite over **white** (TripoSG's reference
51
+ pipeline), not gray.
52
+ - The exported field decoder is **outside-positive** — negate for
53
+ inside-positive marching cubes at iso 0; query bounds ±1.005.
54
+ - Query the decoder in chunks of ≤ 8192 points (cross-attention to 2048 kv
55
+ tokens; huge chunks OOM). Open/release the sessions per stage to keep peak
56
+ memory ~1 GB instead of the >4 GB sum.
57
+ - Output is already +Y-up.
58
+ - **Geometry only** — TripoSG has no colour decoder. QtMeshEditor bakes colour
59
+ by projecting the input photo onto the visible front and filling the rest
60
+ from TripoSR's image-conditioned colour field (see
61
+ [`QtMeshEditor-triposr-onnx`](https://huggingface.co/fernandotonon/QtMeshEditor-triposr-onnx)).
62
+
63
+ Full measured export contract: `docs/TRIPOSG_EXPORT_NOTES.md` in the
64
+ QtMeshEditor repo.
65
+
66
+ ## Reproducing
67
+
68
+ `scripts/export-triposg-onnx.py` in the QtMeshEditor repo (one-time, offline).
69
+
70
+ ## License
71
+
72
+ MIT (same as the upstream code and weights). Credit: VAST-AI-Research.