mnmly/zipsplat-mlx

ZipSplat — MLX weights (fp16 safetensors)

Format conversion of the ZipSplat zipsplat-da3g-252p checkpoint for mlx-swift, used by mlx-swift-ZipSplat.

This is not a new model. It is the original checkpoint re-serialised so it can be loaded on Apple Silicon without PyTorch. All credit for the model belongs to the original authors.

Original work

ZipSplat: Fewer Gaussians, Better Splats — Alexander Veicht, Sunghwan Hong, Dániel Baráth, Marc Pollefeys (ETH Zürich / Microsoft).

• Paper: https://arxiv.org/abs/2606.05102
• Code: https://github.com/cvg/ZipSplat
• Original weights: https://huggingface.co/veichta/zipsplat

Licence

CC BY-NC 4.0 — non-commercial use only. https://creativecommons.org/licenses/by-nc/4.0/

Inherited from the original weights, which carry it because the checkpoint is initialised from DA3-Giant (CC BY-NC 4.0) and trained on DL3DV-10K (CC BY-NC 4.0). The ZipSplat code is Apache-2.0; the weights are not. This conversion is a derivative and carries the same terms.

Changes from the original

zipsplat-da3g-252p.tar (5.79 GB, fp32 PyTorch) → zipsplat-da3g-252p-f16.safetensors (2.90 GB, fp16). 907 tensors, 1.4477 B parameters, verified against the reference model structure with 0 missing and 0 unexpected keys. Three mechanical changes, no retraining and no architectural modification:

1. fp16 cast. Storage only; the port loads at whatever dtype the caller asks for.
2. Two structural key remaps. The ViT's patch_embed.*, cls_token and pos_embed are nested under an embeddings. prefix, matching the module tree in mlx-swift-da3.
3. Two Conv2d transposes. Both patch-embed weights go NCHW → NHWC (0,2,3,1), as MLX convolutions are channels-last.

Reproduce with Scripts/convert_weights.py.

Fidelity

The port was checked against the PyTorch reference at three levels:

check	result
per-stage activations (patch embed → backbone → fuse → head)	within fp16 tolerance
end-to-end .ply, every Gaussian parameter	worst field mean-rel 0.043, all corr ≥ 0.9996
novel views rendered through gsplat's CUDA rasteriser	mean PSNR 46.10 dB, worst 38.23 dB

For scale, the model's own eval PSNR against ground truth is 21.77 dB, so the conversion's deviation sits about 24 dB below the model's own error.

Usage

import MLXZipSplat

let session = try ZipSplatSession(weights: weightsURL)
session.loadViews(images)
let gaussians = session.gaussians(compression: 1.0)[0]
try gaussians.writePLY(to: outputURL)

See mlx-swift-ZipSplat for the CLI and the SwiftUI viewer.