Update README.md
Browse files
README.md
CHANGED
|
@@ -66,7 +66,6 @@ The v0.1 model was tuned toward a deliberately narrow domain to validate the app
|
|
| 66 |
- **Semi-static establishing city / urban scenes** (no heavy camera motion)
|
| 67 |
- **~100° horizontal field of view** in the source clip
|
| 68 |
- **2.39:1 source aspect** (standard cinemascope)
|
| 69 |
-
- **1024×512 @ 24 fps, 41 frames** at inference
|
| 70 |
|
| 71 |
It will **generalize poorly** outside these conditions — fast action, extreme close-ups, heavily
|
| 72 |
stylised imagery, or very different FOVs are not reliably handled yet.
|
|
@@ -87,16 +86,11 @@ Three sample clips (`clip1`, `clip2`, `clip3`) are included under `samples/`.
|
|
| 87 |
|
| 88 |
Load on top of `ltx-2.3-22b-dev.safetensors` with the LTX-2 `video_to_video` pipeline and pass:
|
| 89 |
|
| 90 |
-
- **Trigger word**: `equirectangular` (
|
| 91 |
- **Reference video**: your source clip projected into the equirect canvas with unknown regions masked
|
| 92 |
-
- **Resolution**:
|
| 93 |
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
- LoRA strength: **1.0**
|
| 97 |
-
- Guidance scale (CFG): **4.0**
|
| 98 |
-
- STG scale: **1.0**, blocks `[29]`, mode `stg_v`
|
| 99 |
-
- Inference steps: **20–30**
|
| 100 |
|
| 101 |
### Companion tooling
|
| 102 |
|
|
@@ -123,9 +117,9 @@ repo shows the exact wiring.
|
|
| 123 |
The final **step 3500** checkpoint is shipped here. Intermediate checkpoints were used for
|
| 124 |
validation during training but aren't included in this release.
|
| 125 |
|
| 126 |
-
## What's next
|
| 127 |
|
| 128 |
-
|
| 129 |
|
| 130 |
- Broader subject matter (interiors, landscapes, crowds, vehicles, …)
|
| 131 |
- Varied input FOVs and focal lengths
|
|
@@ -140,6 +134,11 @@ v0.2 is planned on a significantly larger and more diverse dataset (thousands of
|
|
| 140 |
- Outputs are not a substitute for natively captured 360 footage — this is a creative
|
| 141 |
re-projection, not a reconstruction
|
| 142 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 143 |
## License
|
| 144 |
|
| 145 |
Apache-2.0. Inherits any base-model conditions from LTX-2.3-22B.
|
|
|
|
| 66 |
- **Semi-static establishing city / urban scenes** (no heavy camera motion)
|
| 67 |
- **~100° horizontal field of view** in the source clip
|
| 68 |
- **2.39:1 source aspect** (standard cinemascope)
|
|
|
|
| 69 |
|
| 70 |
It will **generalize poorly** outside these conditions — fast action, extreme close-ups, heavily
|
| 71 |
stylised imagery, or very different FOVs are not reliably handled yet.
|
|
|
|
| 86 |
|
| 87 |
Load on top of `ltx-2.3-22b-dev.safetensors` with the LTX-2 `video_to_video` pipeline and pass:
|
| 88 |
|
| 89 |
+
- **Trigger word**: `equirectangular` (works without any prompt too, and with a descriptive prompt you can direct the content of outpainted part)
|
| 90 |
- **Reference video**: your source clip projected into the equirect canvas with unknown regions masked
|
| 91 |
+
- **Resolution**: 1920x960, 121 frames, 24 fps
|
| 92 |
|
| 93 |
+
Only tested in ComfyUI with the workflow available in this repo. Please note that the workflow's padding node crops your input footage to 2.39:1, you can select whether it's cut from center, top or bottom. Other aspect ratios will work poorly in this early version.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 94 |
|
| 95 |
### Companion tooling
|
| 96 |
|
|
|
|
| 117 |
The final **step 3500** checkpoint is shipped here. Intermediate checkpoints were used for
|
| 118 |
validation during training but aren't included in this release.
|
| 119 |
|
| 120 |
+
## What's next
|
| 121 |
|
| 122 |
+
Next version is planned on a significantly larger and more diverse dataset covering:
|
| 123 |
|
| 124 |
- Broader subject matter (interiors, landscapes, crowds, vehicles, …)
|
| 125 |
- Varied input FOVs and focal lengths
|
|
|
|
| 134 |
- Outputs are not a substitute for natively captured 360 footage — this is a creative
|
| 135 |
re-projection, not a reconstruction
|
| 136 |
|
| 137 |
+
## Acknowledgements
|
| 138 |
+
|
| 139 |
+
- First training done at ADOS Paris event in collaboration with Cseti, NebSH and S4f3ty_Marc.
|
| 140 |
+
- Advice from oumoumad in IC-LoRA training altogether has been very valuable to me. The workflow included in this release is modified from oumoumad's IC-LoRA workflow.
|
| 141 |
+
|
| 142 |
## License
|
| 143 |
|
| 144 |
Apache-2.0. Inherits any base-model conditions from LTX-2.3-22B.
|