File size: 3,016 Bytes
be167ee | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | ---
license: mit
tags:
- onnx
- image-to-3d
- 3d-generation
- rectified-flow
- diffusion-transformer
- qtmesheditor
library_name: onnx
pipeline_tag: image-to-3d
---
# TripoSG — ONNX export (staged graphs)
**ONNX re-export of [VAST-AI/TripoSG](https://huggingface.co/VAST-AI/TripoSG)**
(SIGGRAPH 2025 — MIT code + MIT weights): a 1.5B-parameter rectified-flow
diffusion transformer over an SDF VAE, single image → high-fidelity 3D
geometry (≈ commercial Tripo 2.0 quality). All credit for the original
weights goes to VAST-AI-Research.
Exported as **four staged graphs** so the flow loop runs in plain host code —
for **[QtMeshEditor](https://github.com/fernandotonon/QtMeshEditor)**
(`qtmesh generate3d --backend triposg`, GUI Backend dropdown, MCP `backend`
arg), local inference via ONNX Runtime + native marching cubes.
> The files QtMeshEditor downloads at runtime live in the shared
> [`fernandotonon/QtMeshEditor-models`](https://huggingface.co/fernandotonon/QtMeshEditor-models)
> repo under `triposg/`. This repo is the standalone model card + mirror.
## Files
| file | role |
|---|---|
| `triposg_image_encoder.onnx` | DINOv2-224 image encoder (mean/std baked in; CFG unconditional = zeros) |
| `triposg_dit_step.onnx` + `.data` | one DiT flow step (fp32, ~5.8 GB external weights) |
| `triposg_vae_latents.onnx` | VAE latent KV-cache graph — run **once** per generation |
| `triposg_vae_decoder.onnx` | per-point SDF field decoder |
An int8 DiT tier exists in the aggregate repo but is **not recommended**: even
per-channel-quantized, the 1.5B DiT degrades to blobs over the 25-step CFG
flow loop, and dynamic-int8 MatMuls are no faster than fp32 on ARM.
## Inference contract
- **Flow loop (host code):** `σᵢ = 1 − i/N`, timestep `1000·σᵢ`, update
`x += (σᵢ − σᵢ₊₁)·v` — note the sign is **opposite** of stock diffusers
`FlowMatchEulerDiscreteScheduler`. CFG as two batch-1 calls, guidance 7.0,
25 steps default.
- Background removal should composite over **white** (TripoSG's reference
pipeline), not gray.
- The exported field decoder is **outside-positive** — negate for
inside-positive marching cubes at iso 0; query bounds ±1.005.
- Query the decoder in chunks of ≤ 8192 points (cross-attention to 2048 kv
tokens; huge chunks OOM). Open/release the sessions per stage to keep peak
memory ~1 GB instead of the >4 GB sum.
- Output is already +Y-up.
- **Geometry only** — TripoSG has no colour decoder. QtMeshEditor bakes colour
by projecting the input photo onto the visible front and filling the rest
from TripoSR's image-conditioned colour field (see
[`QtMeshEditor-triposr-onnx`](https://huggingface.co/fernandotonon/QtMeshEditor-triposr-onnx)).
Full measured export contract: `docs/TRIPOSG_EXPORT_NOTES.md` in the
QtMeshEditor repo.
## Reproducing
`scripts/export-triposg-onnx.py` in the QtMeshEditor repo (one-time, offline).
## License
MIT (same as the upstream code and weights). Credit: VAST-AI-Research.
|