Update README.md

README.md

@@ -1,3 +1,97 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+base_model: Lightricks/LTX-2.3-22B
+library_name: diffusers
+---
+
+# LTX-2.3 — 360° Equirectangular Outpainting IC-LoRA · **v0.1**
+
+**Proof-of-concept** IC-LoRA adapter for [Lightricks/LTX-2.3-22B](https://huggingface.co/Lightricks) that
+outpaints standard widescreen footage into a full **360° equirectangular** projection for immersive/VR viewing.
+
+> This is an **early v0.1 release**. Expect rough edges, limited subject variety, and inconsistent
+> coherence outside the sweet spot described below. A v0.2 with a much larger, more diverse
+> dataset is planned.
+
+---
+
+## What it does
+
+- **Input**: a flat 2.39:1 (cinemascope) clip and a matching equirectangular *reference* (the input
+  projected into the equirect canvas, with the unknown regions left masked/black)
+- **Output**: the model fills in the masked regions, turning the flat shot into a plausible
+  360° equirectangular video that can be viewed in a VR/360 player
+
+Intended for transforming existing live-action or cinematic footage into immersive content.
+
+## Sweet spot (v0.1)
+
+The v0.1 model was tuned toward a deliberately narrow domain to validate the approach:
+
+- **Semi-static establishing city / urban scenes** (no heavy camera motion)
+- **~100° horizontal field of view** in the source clip
+- **2.39:1 source aspect** (standard cinemascope)
+- **1024×512 @ 24 fps, 41 frames** at inference
+
+It will **generalize poorly** outside these conditions — fast action, extreme close-ups, heavily
+stylised imagery, or very different FOVs are not reliably handled yet.
+
+## Usage
+
+Load on top of `ltx-2.3-22b-dev.safetensors` with the LTX-2 `video_to_video` pipeline and pass:
+
+- **Trigger word**: `equirectangular` (required, include in every prompt)
+- **Reference video**: your source clip projected into the equirect canvas with unknown regions masked
+- **Resolution**: 1024×512, 41 frames, 24 fps (other shapes untested)
+
+Recommended starting points:
+
+- LoRA strength: **1.0**
+- Guidance scale (CFG): **4.0**
+- STG scale: **1.0**, blocks `[29]`, mode `stg_v`
+- Inference steps: **20–30**
+
+### Companion tooling
+
+A small ComfyUI helper node — **ComfyUI-EquirectProjector** — was written alongside this LoRA to
+produce the masked equirectangular reference from a flat clip. Pair it with the standard
+LTX-2 video-to-video nodes.
+
+## Training (v0.1)
+
+| | |
+|--|--|
+| Base model | LTX-2.3-22B (dev) |
+| Strategy | IC-LoRA (`video_to_video`) |
+| Rank / alpha | 128 / 128 |
+| Target modules | video self+cross attention + FFN |
+| Resolution | 1024×512, 41 frames @ 24 fps |
+| Optimizer | Prodigy (D-Adaptation), lr=1.0, constant |
+| Precision | bf16, gradient checkpointing |
+| Steps | 3500 |
+| Hardware | 1× NVIDIA H100 80GB |
+| Dataset | Small curated POC set (not released) — semi-static city establishing clips |
+
+Multiple checkpoints are published so you can pick the one that best fits your content:
+`step_00500`, `step_01500`, `step_02700`, `step_03500`.
+
+## What's next (v0.2)
+
+v0.2 is planned on a significantly larger and more diverse dataset (thousands of clips) covering:
+
+- Broader subject matter (interiors, landscapes, crowds, vehicles, …)
+- Varied input FOVs and focal lengths
+- A wider range of camera motion — not just static establishing shots
+- Better handling of the polar regions (top/bottom caps of the equirect canvas)
+
+## Limitations
+
+- Does not model the top/bottom caps of the sphere well — expect stretching or repetition
+- Struggles with busy motion and fast cuts
+- Prompt adherence is weak; conditioning is dominated by the reference video
+- Outputs are not a substitute for natively captured 360 footage — this is a creative
+  re-projection, not a reconstruction
+
+## License
+
+Apache-2.0. Inherits any base-model conditions from LTX-2.3-22B.