gregt commited on
Commit
5f703d3
Β·
verified Β·
1 Parent(s): fa83420

Add model card

Browse files
Files changed (1) hide show
  1. README.md +94 -5
README.md CHANGED
@@ -1,5 +1,94 @@
1
- ---
2
- license: apache-2.0
3
- base_model:
4
- - Lightricks/LTX-2.3
5
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Lightricks/LTX-Video
4
+ tags:
5
+ - ltx-video
6
+ - ltx-2.3
7
+ - lora
8
+ - ic-lora
9
+ - sdr-to-hdr
10
+ - hdr
11
+ - color-grading
12
+ - video-to-video
13
+ - logc3
14
+ pipeline_tag: video-to-video
15
+ library_name: diffusers
16
+ ---
17
+
18
+ # LTX-2.3 SDR β†’ HDR IC-LoRA
19
+
20
+ A LoRA adapter for **LTX-Video 2.3 (22B)** that converts SDR video into HDR (LogC3 encoded), trained as an **IC-LoRA** (in-context LoRA) using the `video_to_video` strategy from `ltxv-trainer`.
21
+
22
+ The model takes an SDR clip as a conditioning reference and generates a matching HDR (LogC3) version, suitable for grading downstream in DaVinci Resolve, Baselight, or Nuke.
23
+
24
+ Inspired by Lightricks' **LumiVid** paper ([arXiv:2604.11788](https://arxiv.org/abs/2604.11788)) β€” same core idea (LogC3 target, IC-LoRA-style reference conditioning, ~10K steps).
25
+
26
+ ## Checkpoint
27
+
28
+ `lora_weights_step_07000.safetensors` β€” step 7,000 of a planned 10,000-step run.
29
+
30
+ ## Usage (ComfyUI)
31
+
32
+ Use `LTXICLoRALoaderModelOnly` from [Lightricks/ComfyUI-LTXVideo](https://github.com/Lightricks/ComfyUI-LTXVideo):
33
+
34
+ 1. Load LTX-2.3 base model
35
+ 2. Load this LoRA via `LTXICLoRALoaderModelOnly`
36
+ 3. Connect your SDR clip as the **reference video**
37
+ 4. Run the IC-LoRA pipeline β€” Stage 1 (low-res, LoRA active) β†’ Stage 2 (upsample, no LoRA)
38
+
39
+ **No trigger word** β€” the LoRA is always active when loaded.
40
+
41
+ **Recommended CFG:** up to **1.5** works well.
42
+
43
+ ## Training Details
44
+
45
+ | | |
46
+ |---|---|
47
+ | Base model | LTX-Video 2.3 22B (`ltx-2.3-22b-dev`) |
48
+ | Text encoder | Gemma 3 12B |
49
+ | Strategy | IC-LoRA (`video_to_video` in [ltxv-trainer](https://github.com/Lightricks/LTX-Video-Trainer)) |
50
+ | LoRA rank / alpha | 32 / 32 |
51
+ | Target modules | `attn1/2.to_{k,q,v,out.0}`, `ff.net.0.proj`, `ff.net.2` |
52
+ | Resolution | 1280 Γ— 736 |
53
+ | Frames per clip | 49 |
54
+ | Batch size | 1 |
55
+ | Learning rate | 2e-4, cosine schedule |
56
+ | Steps | 7,000 (target 10,000) |
57
+ | Precision | bf16 + gradient checkpointing |
58
+ | Log curve | **LogC3** (validated optimal via KL divergence in LumiVid) |
59
+
60
+ ## Dataset
61
+
62
+ - **200 clips** rendered from PolyHaven HDRIs with virtual camera moves
63
+ - Paired SDR ↔ LogC3 HDR clips
64
+ - SDR side intentionally degraded (compression, blur, contrast, white-balance shift) to mimic real-world camera capture
65
+
66
+ ## Intended Use
67
+
68
+ - Converting SDR (Rec.709) video to LogC3 HDR for further grading
69
+ - VFX / DI pipelines that already speak LogC3 (Nuke, Baselight, DaVinci Resolve)
70
+ - Outputs are designed to be exported to ProRes 4444 or OpenEXR sequences
71
+
72
+ ## Limitations
73
+
74
+ - Trained primarily on PolyHaven HDRI environments β€” limited human/motion diversity in v1
75
+ - Output is **LogC3, not display-referred HDR** β€” you still need a grading pass to map to PQ/HLG/Rec.2100
76
+ - The model can do a surprisingly good job of *hallucinating* believable detail back into clipped highlights (skies, practicals, blown-out windows), but this is generative reconstruction β€” it is not faithful recovery of the original photons. Use with eyes open in any pipeline where photographic accuracy matters.
77
+ - Inference is two-stage (low-res β†’ upsample); expect LTX-2.3's typical compute footprint
78
+
79
+ ## Citation
80
+
81
+ If this is useful, please reference the LumiVid paper that inspired the approach:
82
+
83
+ ```
84
+ @article{lumivid2026,
85
+ title={LumiVid: Distilling LDR Video Diffusion Models for HDR Video Generation},
86
+ author={Lightricks},
87
+ journal={arXiv:2604.11788},
88
+ year={2026}
89
+ }
90
+ ```
91
+
92
+ ## License
93
+
94
+ Apache 2.0. Base model license (LTX-Video) applies to inference use.