VR-360-Outpaint 1.0 — LTX-2.3 IC-LoRA
THE 1.0 IS FINALLY OUT!
360° equirectangular video outpainting. Give it a flat (converted to inverse gnomonic by my nodes) video crop on a black equirectangular canvas, and it fills the unknown regions to produce a full 360° equirectangular video suitable for VR / immersive playback.
Built on Lightricks/LTX-2.3-22B via IC-LoRA (video-to-video conditioning). This is the production "v1.0" run — a fully retrained IC-LoRA, multi-reference augmentation, and mixed-resolution buckets.
How it works: the inverse gnomonic projection trick
The core insight behind VR-360-Outpaint — first developed in April 2026 by yours truly — is that a video diffusion model can struggle with equirectangular geometry. Asking it to "extend this flat crop into 360°" without geometric guidance is a near-impossible task: the model must simultaneously understand both the spherical projection mapping and the outpainting task.
The solution that I came up with was to take the rectilinear input video and apply an inverse gnomonic projection, compositing it onto the equirectangular canvas at the correct angular position and field of view. This gives the model an explicit geometric nudge — it can see exactly where and how the known pixels sit on the sphere, and what the local rectilinear perspective looks like in spherical space. The outpainting task then becomes: 'given this correctly mapped window onto the sphere, complete the rest of the equirectangular frame in a way that is consistent with it.'
This geometric conditioning is applied identically in both training and inference — every training pair was a 360° video paired with a gnomonic crop of a specific FOV and aspect ratio on the canvas, and the model learned to recover the full equirectangular frame from that partial view. At inference time, the ComfyUI-VR-Outpaint-Tools node pack performs the same projection from your flat footage, now also including automatic FOV estimation via the GeoCalib node.
Without this geometric scaffolding, the model would have to discover spherical projection rules from its core training — although LTX2.3 obviously has some equirectangular material in its core training data, this still presents a vastly harder learning problem. The gnomonic projection provides the inductive bias that makes VR outpainting tractable.
Quick start
- Base model:
Lightricks/LTX-2.3-22B(dev safetensors) - Load this LoRA on top via the LTX-2
video_to_videopipeline (ComfyUI recommended) - Trigger word:
equirectangular - Caption (the only one the model knows):
360 degree equirectangular panorama, high quality, detailed - Resolution: 1920×960, 121 frames @ 24 fps
- Workflow: Use the included
Burgstall-VR-Outpaint.jsonin ComfyUI - Companion tools: ComfyUI-VR-Outpaint-Tools provides the gnomonic projection nodes and the GeoCalib FOV estimation node for automatic field-of-view detection from your flat footage — no manual FOV entry needed.
Examples
Each example shows the flat input video, a slow gnomonic pan across the outpainted 360° result (a virtual camera sweeping through the scene), and the raw equirectangular output.
| Flat input | Pan across the result | Raw equirectangular output |
Which checkpoint?
Three checkpoints are provided (.safetensors, ~1.3 GB each). They all produce valid equirectangular
output; pick based on your priorities:
| Checkpoint | Step | Character |
|---|---|---|
360vroutpaint_v2_step07000.safetensors |
7000 | Recommended. Best balance of equirectangular consistency and scene variety |
360vroutpaint_v2_step05000.safetensors |
5000 | Slightly more creative outpainting, but the 360° wrap can have a more prominent seam |
360vroutpaint_v2_step09000.safetensors |
9000 | Tightest equirectangular consistency, but tends toward more uniform / monotonic scenery |
All three are fully converged. The differences are subtle; if unsure, start with step 7000.
Training
| Parameter | Value |
|---|---|
| Base model | LTX-2.3-22B (dev) |
| Strategy | IC-LoRA (video_to_video, first_frame_conditioning_p: 0.0) |
| LoRA rank / alpha | 128 / 128 |
| Target modules | Self-attn (q, k, v, out) + cross-attn (q, k, v, out) + FFN (ff.net.0.proj, ff.net.2) |
| Trainable params | 654M |
| Resolution | Mixed buckets: 1024×512×121 + 1920×960×49 |
| Frame rate | 24 fps |
| Optimizer | Prodigy (D-Adaptation), lr=1.0, decoupled weight decay, constant schedule |
| Precision | bf16 (weights + Gemma text encoder), gradient checkpointing ON |
| Batch / accum | 1 / 1 |
| Steps trained | 9000 (converged by ~3000, stopped on plateau) |
| Hardware | 1× RTX PRO 6000 Blackwell (96 GB), compute sponsored by Lightricks |
| Caption | 360 degree equirectangular panorama, high quality, detailed |
Dataset
The model was trained on 13,281 pairs, sampled from a master dataset of 53,124 augmented video pairs built from 4,427 unique 360° clips sourced from 51 CC-BY YouTube channels.
Each of the 4,427 clips was rendered into 12 reference variants — the combination of:
- 4 fields of view: 70°, 90°, 110°, 130°
- 3 aspect ratios: 1.33 (4:3), 1.78 (16:9), 2.39 (cinemascope)
From these 12 variants per clip, 3 distinct FOVs were assigned per target (globally balanced across all 12 FOV×aspect cells, ~1,107 pairs each), and the trainer randomly sampled one of the 3 at each step. A per-clip random yaw shift (0–1920 px) exploits equirectangular rotational symmetry, effectively multiplying each scene into many independent viewpoints and keeping operators / tripods off a fixed azimuth.
All clips in the master dataset were normalized to 1920×960 at 24 fps, 121 frames (~5 seconds).
The training dataset is private and not distributed with these weights.
Validation
153 held-out clips from 6 diverse non-CC sources (urban day/night, indoor, nature, action, aurora). Eval-only — never trained on. Validation is sample-generation only (no held-out val-loss exists).
Convergence evidence
Loss is a poor signal for IC-LoRA. I tracked three independent metrics; all agree the model converged by step 3000 and plateaued through 9000:
1. 360° seam continuity (seam_ratio — wrap-edge MSE ÷ mean adjacent-column MSE, lower is better):
| Step | Seam ratio |
|---|---|
| 1000 | 59.7 |
| 2000 | 33.4 |
| 3000 | 6.1 ← converged |
| 5000 | 9.3 |
| 7000 | 5.1 |
| 9000 | 5.6 |
The wrap seam dropped from ~60 to ~6 in 3000 steps and stayed in the 5–9 band (noise floor) through 9000. The residual ~5–6 is a data/config limit, not fixable by more steps.
2. Weight delta (relative ‖Wₙ − Wₙ₋₅₀₀‖):
- Early: 0.15–0.17 (high, noisy learning)
- Step 5000: 0.137
- Step 7500+: 0.126–0.135 (stable plateau — no continued learning)
3. Visual inspection: Side-by-side comparisons at steps 5000, 7000, and 9000 are visually near-identical for most scenes, with mostly just subtle sharpening differences as well as negligible differences in seam.
Sweet spot
The model was tuned on and performs best with:
- Content: Semi-static establishing shots — cityscapes, urban scenes, landscapes
- FOV: ~90–110° horizontal (the training distribution center)
- Aspect ratio: 1.78–2.39 (standard widescreen to cinemascope)
- Camera motion: Slow pans, gentle dolly, static tripod
It will generalize beyond these conditions — that's why I trained across 4 FOVs and 3 aspect ratios — but quality degrades gracefully as you move further from the training distribution.
Limitations
- Top & bottom caps (zenith / nadir of the sphere) are the hardest regions; the model has less signal there. Avoid critical detail near the poles.
- Busy motion (fast cuts, whip pans, handheld shake) easily break the wrap seam.
- Prompt adherence — conditioning is dominated by the reference video, not text. The base caption / prompt is good to keep there, but you can describe the elements that you want there to be in the outpainted area to guide it. It will outpaint with the base prompt only, too.
- Narrow domain — trained on outdoor / architectural scenes. Interiors, crowds, close-ups, and organic subjects might work less reliably, although these draw strongly from the model's base training.
Companion tools
- ComfyUI-VR-Outpaint-Tools —
A ComfyUI node pack providing:
- Gnomonic projection nodes for converting flat footage into the masked equirectangular reference this LoRA expects (FOV selection, projection, canvas compositing)
- GeoCalib FOV estimation node for automatic field-of-view detection from your footage — no manual measurement needed
- Formerly named ComfyUI-EquirectProjector; renamed after expanding scope beyond the original idea of just forward gnomonic projection.
Burgstall-VR-Outpaint.json(included in this repo) — End-to-end ComfyUI workflow wiring the tools → LTX-2.3 + this LoRA → output.
⚠️ Repo re-star request: The node pack was renamed from
ComfyUI-EquirectProjectortoComfyUI-VR-Outpaint-Tools, and in the process I accidentally turned the repo private for a moment — which resets the star count to zero. If you starred it before, I'd really appreciate it if you could drop by and re-star it if you appreciate this work. Every one helps! ⭐
What's next
I am currently in the process of developing a seamless equirectangular video generation method. Stay tuned.
Acknowledgements
- Compute sponsorship by Lightricks — the LTX team graciously provided the GPU hardware for this training run. Special thanks to Ofir Bibi for technical guidance and advice throughout the project.
- Initial PoC training done at ADOS Paris 2026 with collaborators Cseti, NebSH, and S4f3ty_Marc. Thank you for your collaboration and support.
- IC-LoRA training methodologies and multi-reference dataset design advised by oumoumad.
- Built on the Lightricks LTX-2.3 foundation model.
Training data attribution
The model was trained on 360° video clips from the following YouTube creators, all shared under Creative Commons Attribution (CC-BY) license. Thank you to each of them for making their work available for reuse.
| Creator | Clips | Channel |
|---|---|---|
| Luxury Travel VR | 5 | @LuxuryTravelVR |
| Watchers Club | 4 | @watchersclub |
| Naomi Wu | 3 | @Naomi-Wu |
| whereisnoa | 3 | @whereisnoa |
| Self-following 360° shooting [SNOWBIRD] | 2 | @360snowbird |
| Travel Smart Seniors | 2 | @travelsmartseniors |
| HaoRyu32 | 2 | @haoryu3233 |
| VR Tours 360 | 2 | @VRTours360 |
| 360VR Tours | 2 | @360vr_Tours |
| PAV360 VR tour | 2 | @PAV360 |
| Snicel 69 | 2 | @Snicel69 |
| Heyman Chan | 1 | @heymanchan7829 |
| Hike360 | 1 | @Hike360 |
| TokyoNoob | 1 | @TokyoNoob |
| Switchflick Productions | 1 | @SwitchflickProductions |
| Dan Lockhart | 1 | @DanLockhart1 |
| 360° Attractions — Immersive Experiences | 1 | @FUNFAIRRIDES360 |
| Tucson Arizona Real Estate | 1 | @tucsonarizonarealestate |
| LearningNowHere sg | 1 | @learningnowheresg4316 |
| VR 360 Vídeo | 1 | @vr360video2 |
| Captain Steve Plays Vlogs Talks | 1 | @CaptainStevePlaysVlogsTalks |
| Eduard Moraru | 1 | @Enygma2002 |
| Day1 Walks | 1 | @day1_walks |
| Alliks82 | 1 | @Alliks82 |
| BigT 360 | 1 | @BigT_360 |
| PIXEO VR | 1 | @pixeovr |
| Real Madrid | 1 | @realmadrid |
| Jonathan Sadler | 1 | @jonnysadler4064 |
| N Flight Video | 1 | @nflightvideo7787 |
| This Way Up Travel | 1 | @ThisWayUpTravel |
| Top Virtual Tours | 1 | @topvirtualtours |
| Train H | 1 | @Train_H |
| He Spoke Style | 1 | @hespokestyle |
| 3D-VR-360 Videos | 1 | @D-VR-VIDEOSREALIDADVIRTUAL |
| Virtual Trip | 1 | @virtualtrip4316 |
| MsBlackoreanlady | 1 | @MsBlackoreanlady |
| Frontline Ulster | 1 | @frontlineulster |
| Radu Ciocan | 1 | @RaduCiocan |
| Pogiboy Productions | 1 | @PogiboyProductions |
| AlexTravel360 | 1 | @AlexTravel360 |
| Glen Park | 1 | @o2warp |
| Joni Gritzner | 1 | @jonigr |
| IDEAS STUDIO | 1 | @IDEASVRTV |
| DroneTourism507 | 1 | @dronetourism507 |
| Duncan Schieb | 1 | @Haveabeerwithduncan |
| Tony McIlwain | 1 | @TonyMcIlwain |
All clips were obtained under CC-BY as indicated by YouTube at the time of download. License proof
(.info.json sidecar files) is archived.
VR-360-Outpaint 1.0 — License
This model is released under Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0).
Commercial licensing
The CC BY-NC 4.0 license permits use for non-commercial purposes only. If you would like to use VR-360-Outpaint in a commercial product, service, or workflow, please contact me for a commercial license: howdy@theaiwrangler.com