README.md

metadata

license: mit
tags:
  - onnx
  - image-to-3d
  - 3d-generation
  - rectified-flow
  - diffusion-transformer
  - qtmesheditor
library_name: onnx
pipeline_tag: image-to-3d

TripoSG — ONNX export (staged graphs)

ONNX re-export of VAST-AI/TripoSG (SIGGRAPH 2025 — MIT code + MIT weights): a 1.5B-parameter rectified-flow diffusion transformer over an SDF VAE, single image → high-fidelity 3D geometry (≈ commercial Tripo 2.0 quality). All credit for the original weights goes to VAST-AI-Research.

Exported as four staged graphs so the flow loop runs in plain host code — for QtMeshEditor (qtmesh generate3d --backend triposg, GUI Backend dropdown, MCP backend arg), local inference via ONNX Runtime + native marching cubes.

The files QtMeshEditor downloads at runtime live in the shared fernandotonon/QtMeshEditor-models repo under triposg/. This repo is the standalone model card + mirror.

Files

file	role
triposg_image_encoder.onnx	DINOv2-224 image encoder (mean/std baked in; CFG unconditional = zeros)
triposg_dit_step.onnx + .data	one DiT flow step (fp32, ~5.8 GB external weights)
triposg_vae_latents.onnx	VAE latent KV-cache graph — run once per generation
triposg_vae_decoder.onnx	per-point SDF field decoder

An int8 DiT tier exists in the aggregate repo but is not recommended: even per-channel-quantized, the 1.5B DiT degrades to blobs over the 25-step CFG flow loop, and dynamic-int8 MatMuls are no faster than fp32 on ARM.

Inference contract

• Flow loop (host code): σᵢ = 1 − i/N, timestep 1000·σᵢ, update x += (σᵢ − σᵢ₊₁)·v — note the sign is opposite of stock diffusers FlowMatchEulerDiscreteScheduler. CFG as two batch-1 calls, guidance 7.0, 25 steps default.
• Background removal should composite over white (TripoSG's reference pipeline), not gray.
• The exported field decoder is outside-positive — negate for inside-positive marching cubes at iso 0; query bounds ±1.005.
• Query the decoder in chunks of ≤ 8192 points (cross-attention to 2048 kv tokens; huge chunks OOM). Open/release the sessions per stage to keep peak memory ~1 GB instead of the >4 GB sum.
• Output is already +Y-up.
• Geometry only — TripoSG has no colour decoder. QtMeshEditor bakes colour by projecting the input photo onto the visible front and filling the rest from TripoSR's image-conditioned colour field (see QtMeshEditor-triposr-onnx).

Full measured export contract: docs/TRIPOSG_EXPORT_NOTES.md in the QtMeshEditor repo.

Reproducing

scripts/export-triposg-onnx.py in the QtMeshEditor repo (one-time, offline).

License

MIT (same as the upstream code and weights). Credit: VAST-AI-Research.