Hugging Face's logo

TheBurgstall
/
VR-360-Outpaint-LTX2.3-IC-LoRA

Video-to-Video
ltx-2.3
ic-lora
lora
360
equirectangular
vr
outpainting
video
Model card Files
xet
 Community
1
VR-360-Outpaint-LTX2.3-IC-LoRA / README.md
TheBurgstall's picture
TheBurgstall
v0.1: weights + ComfyUI workflow + sample clips
f476618 verified
preview code
|
Raw
History Blame
4.57 kB
metadata 
license: apache-2.0
base_model: Lightricks/LTX-2.3-22B
library_name: diffusers

LTX-2.3 — 360° Equirectangular Outpainting IC-LoRA · v0.1

Proof-of-concept IC-LoRA adapter for Lightricks/LTX-2.3-22B that outpaints standard widescreen footage into a full 360° equirectangular projection for immersive/VR viewing.

This is an early v0.1 release. Expect rough edges, limited subject variety, and inconsistent coherence outside the sweet spot described below. A v0.2 with a much larger, more diverse dataset is planned.

What it does

  • Input: a flat 2.39:1 (cinemascope) clip and a matching equirectangular reference (the input projected into the equirect canvas, with the unknown regions left masked/black)
  • Output: the model fills in the masked regions, turning the flat shot into a plausible 360° equirectangular video that can be viewed in a VR/360 player

Intended for transforming existing live-action or cinematic footage into immersive content.

Sweet spot (v0.1)

The v0.1 model was tuned toward a deliberately narrow domain to validate the approach:

  • Semi-static establishing city / urban scenes (no heavy camera motion)
  • ~100° horizontal field of view in the source clip
  • 2.39:1 source aspect (standard cinemascope)
  • 1024×512 @ 24 fps, 41 frames at inference

It will generalize poorly outside these conditions — fast action, extreme close-ups, heavily stylised imagery, or very different FOVs are not reliably handled yet.

Files

File What it is
ltx-2.3-22b-ic-lora-360-equirect-poc-step3500.safetensors The LoRA weights (final step 3500 checkpoint, ~1.3 GB)
Equirect-Outpaint.json Reference ComfyUI workflow wired end-to-end for this LoRA
samples/clipN-fl-eq.mp4 Flat input + equirect output side-by-side (3626×960)
samples/clipN-eq.mp4 Raw equirectangular output (1920×960) — load in a 360° player
samples/clipN-sweep.mp4 2D camera sweep through the 360° output (1920×1080) for quick preview without a VR player

Three sample clips (clip1, clip2, clip3) are included under samples/.

Usage

Load on top of ltx-2.3-22b-dev.safetensors with the LTX-2 video_to_video pipeline and pass:

  • Trigger word: equirectangular (required, include in every prompt)
  • Reference video: your source clip projected into the equirect canvas with unknown regions masked
  • Resolution: 1024×512, 41 frames, 24 fps (other shapes untested)

Recommended starting points:

  • LoRA strength: 1.0
  • Guidance scale (CFG): 4.0
  • STG scale: 1.0, blocks [29], mode stg_v
  • Inference steps: 20–30

Companion tooling

A small ComfyUI helper pack — ComfyUI-EquirectProjector — was written alongside this LoRA to produce the masked equirectangular reference from a flat clip. Pair it with the standard LTX-2 video-to-video nodes. The Equirect-Outpaint.json workflow in this repo shows the exact wiring.

Training (v0.1)

Base model LTX-2.3-22B (dev)
Strategy IC-LoRA (video_to_video)
Rank / alpha 128 / 128
Target modules video self+cross attention + FFN
Resolution 1024×512, 41 frames @ 24 fps
Optimizer Prodigy (D-Adaptation), lr=1.0, constant
Precision bf16, gradient checkpointing
Steps 3500
Hardware 1× NVIDIA H100 80GB
Dataset Small curated POC set (not released) — semi-static city establishing clips

The final step 3500 checkpoint is shipped here. Intermediate checkpoints were used for validation during training but aren't included in this release.

What's next (v0.2)

v0.2 is planned on a significantly larger and more diverse dataset (thousands of clips) covering:

  • Broader subject matter (interiors, landscapes, crowds, vehicles, …)
  • Varied input FOVs and focal lengths
  • A wider range of camera motion — not just static establishing shots
  • Better handling of the polar regions (top/bottom caps of the equirect canvas)

Limitations

  • Does not model the top/bottom caps of the sphere well — expect stretching or repetition
  • Struggles with busy motion and fast cuts
  • Prompt adherence is weak; conditioning is dominated by the reference video
  • Outputs are not a substitute for natively captured 360 footage — this is a creative re-projection, not a reconstruction

License

Apache-2.0. Inherits any base-model conditions from LTX-2.3-22B.