Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
tags:
|
| 4 |
+
- onnx
|
| 5 |
+
- image-to-3d
|
| 6 |
+
- 3d-generation
|
| 7 |
+
- rectified-flow
|
| 8 |
+
- diffusion-transformer
|
| 9 |
+
- qtmesheditor
|
| 10 |
+
library_name: onnx
|
| 11 |
+
pipeline_tag: image-to-3d
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# TripoSG — ONNX export (staged graphs)
|
| 15 |
+
|
| 16 |
+
**ONNX re-export of [VAST-AI/TripoSG](https://huggingface.co/VAST-AI/TripoSG)**
|
| 17 |
+
(SIGGRAPH 2025 — MIT code + MIT weights): a 1.5B-parameter rectified-flow
|
| 18 |
+
diffusion transformer over an SDF VAE, single image → high-fidelity 3D
|
| 19 |
+
geometry (≈ commercial Tripo 2.0 quality). All credit for the original
|
| 20 |
+
weights goes to VAST-AI-Research.
|
| 21 |
+
|
| 22 |
+
Exported as **four staged graphs** so the flow loop runs in plain host code —
|
| 23 |
+
for **[QtMeshEditor](https://github.com/fernandotonon/QtMeshEditor)**
|
| 24 |
+
(`qtmesh generate3d --backend triposg`, GUI Backend dropdown, MCP `backend`
|
| 25 |
+
arg), local inference via ONNX Runtime + native marching cubes.
|
| 26 |
+
|
| 27 |
+
> The files QtMeshEditor downloads at runtime live in the shared
|
| 28 |
+
> [`fernandotonon/QtMeshEditor-models`](https://huggingface.co/fernandotonon/QtMeshEditor-models)
|
| 29 |
+
> repo under `triposg/`. This repo is the standalone model card + mirror.
|
| 30 |
+
|
| 31 |
+
## Files
|
| 32 |
+
|
| 33 |
+
| file | role |
|
| 34 |
+
|---|---|
|
| 35 |
+
| `triposg_image_encoder.onnx` | DINOv2-224 image encoder (mean/std baked in; CFG unconditional = zeros) |
|
| 36 |
+
| `triposg_dit_step.onnx` + `.data` | one DiT flow step (fp32, ~5.8 GB external weights) |
|
| 37 |
+
| `triposg_vae_latents.onnx` | VAE latent KV-cache graph — run **once** per generation |
|
| 38 |
+
| `triposg_vae_decoder.onnx` | per-point SDF field decoder |
|
| 39 |
+
|
| 40 |
+
An int8 DiT tier exists in the aggregate repo but is **not recommended**: even
|
| 41 |
+
per-channel-quantized, the 1.5B DiT degrades to blobs over the 25-step CFG
|
| 42 |
+
flow loop, and dynamic-int8 MatMuls are no faster than fp32 on ARM.
|
| 43 |
+
|
| 44 |
+
## Inference contract
|
| 45 |
+
|
| 46 |
+
- **Flow loop (host code):** `σᵢ = 1 − i/N`, timestep `1000·σᵢ`, update
|
| 47 |
+
`x += (σᵢ − σᵢ₊₁)·v` — note the sign is **opposite** of stock diffusers
|
| 48 |
+
`FlowMatchEulerDiscreteScheduler`. CFG as two batch-1 calls, guidance 7.0,
|
| 49 |
+
25 steps default.
|
| 50 |
+
- Background removal should composite over **white** (TripoSG's reference
|
| 51 |
+
pipeline), not gray.
|
| 52 |
+
- The exported field decoder is **outside-positive** — negate for
|
| 53 |
+
inside-positive marching cubes at iso 0; query bounds ±1.005.
|
| 54 |
+
- Query the decoder in chunks of ≤ 8192 points (cross-attention to 2048 kv
|
| 55 |
+
tokens; huge chunks OOM). Open/release the sessions per stage to keep peak
|
| 56 |
+
memory ~1 GB instead of the >4 GB sum.
|
| 57 |
+
- Output is already +Y-up.
|
| 58 |
+
- **Geometry only** — TripoSG has no colour decoder. QtMeshEditor bakes colour
|
| 59 |
+
by projecting the input photo onto the visible front and filling the rest
|
| 60 |
+
from TripoSR's image-conditioned colour field (see
|
| 61 |
+
[`QtMeshEditor-triposr-onnx`](https://huggingface.co/fernandotonon/QtMeshEditor-triposr-onnx)).
|
| 62 |
+
|
| 63 |
+
Full measured export contract: `docs/TRIPOSG_EXPORT_NOTES.md` in the
|
| 64 |
+
QtMeshEditor repo.
|
| 65 |
+
|
| 66 |
+
## Reproducing
|
| 67 |
+
|
| 68 |
+
`scripts/export-triposg-onnx.py` in the QtMeshEditor repo (one-time, offline).
|
| 69 |
+
|
| 70 |
+
## License
|
| 71 |
+
|
| 72 |
+
MIT (same as the upstream code and weights). Credit: VAST-AI-Research.
|