Hugging Face's logo

Lightricks
/
LTX-2.3-22b-IC-LoRA-Day-To-Night

Video-to-Video
English
ltx-video
ic-lora
ltx-2.3
day-to-night
relighting
Model card Files
xet
 Community
LTX-2.3-22b-IC-LoRA-Day-To-Night / README.md
art-alex's picture
art-alex
Update README.md
733de9d verified
preview code
|
Raw
History Blame Contribute Delete
8.11 kB
metadata 
base_model:
  - Lightricks/LTX-2.3
base_model_relation: adapter
license: other
license_name: ltx-2-community-license
license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE
language:
  - en
tags:
  - ltx-video
  - ic-lora
  - ltx-2.3
  - video-to-video
  - day-to-night
  - relighting
pipeline_tag: video-to-video
extra_gated_description: >-
  By clicking "Agree and Access" you acknowledge the [Privacy
  Policy](https://static.lightricks.com/legal/Privacy%20Policy%20-%20LTX%20Platform.pdf) 
  and consent to receive offers and updates. You can unsubscribe at any time.
extra_gated_button_content: Agree and Access
widget:
  - text: >-
      A realistic nighttime outdoor scene. A mountain landscape under a dark sky
      filled with bright stars and a prominent Milky Way. Deep natural shadows
      across the terrain with very faint ambient starlight. Photorealistic
      night. Only the lighting changes from day to night; identical composition,
      framing, camera movement and motion.
    output:
      url: examples/Examples_6__step_003000_02.mp4
  - text: >-
      A realistic nighttime scene. A cafe interior looking out at a dark city
      street at night. Warm overhead interior lights cast amber reflections on
      the furniture, contrasting with the cool ambient streetlights and deep
      shadows outside. Only the lighting changes from day to night; identical
      composition, framing, camera movement and motion.
    output:
      url: examples/p_ind_35318605_night.mp4
  - text: >-
      A realistic nighttime scene. A dimly lit interior at night with a warm,
      low-intensity artificial light coming from the right, casting deep shadows
      across the room and soft amber highlights on the subject reading. Low-key
      lighting, high contrast. Only the lighting changes from day to night;
      identical composition, framing, camera movement and motion.
    output:
      url: examples/p_ind_6799040_night.mp4

LTX-2.3 22B IC-LoRA Day-to-Night Relighting

This is a Day-to-Night Relighting IC-LoRA trained on top of LTX-2.3-22B, which re-renders a daytime video as the same shot at night while preserving composition, framing, camera movement, and subject motion frame-for-frame.

It is based on the LTX-2.3 foundation model.

Example Outputs

Prompt
A realistic nighttime outdoor scene. A mountain landscape under a dark sky filled with bright stars and a prominent Milky Way. Deep natural shadows across the terrain with very faint ambient starlight. Photorealistic night. Only the lighting changes from day to night; identical composition, framing, camera movement and motion.
Prompt
A realistic nighttime scene. A cafe interior looking out at a dark city street at night. Warm overhead interior lights cast amber reflections on the furniture, contrasting with the cool ambient streetlights and deep shadows outside. Only the lighting changes from day to night; identical composition, framing, camera movement and motion.
Prompt
A realistic nighttime scene. A dimly lit interior at night with a warm, low-intensity artificial light coming from the right, casting deep shadows across the room and soft amber highlights on the subject reading. Low-key lighting, high contrast. Only the lighting changes from day to night; identical composition, framing, camera movement and motion.

Model Files

ltx-2.3-22b-ic-lora-day-to-night-0.9.safetensors

The recommended default and only shipped checkpoint (final, step 3000).

Model Details

  • Base Model: LTX-2.3-22B Video
  • Training Type: IC-LoRA (video-to-video)
  • Control Type: Daytime reference video — the model conditions on a day clip and relights it to night.
  • Reference Downscale Factor: 1 (the reference video is processed at the same resolution as the output).
  • Pipeline details: No special pre/post-processing. The daytime reference is resized + center-cropped to the target resolution at inference; outputs are generated directly in pixel space via the VAE.

Intended Use & Out-of-Scope

Intended use: Converting short real-world day videos into a photorealistic nighttime version of the same shot, keeping motion and layout intact (e.g. outdoor scenes, landscapes, streets, people in motion). Best at the trained resolutions and ~4s (97 frames) clip length.

Out of scope: Inventing new scenes or camera moves, stylized/non-photoreal looks, and clips much longer than the training length (longer references can drift toward the end). Indoor and heavily artificial-light scenes work but are outside the primary training distribution.

Control Signal Requirements

  • Control signal type: Daytime video (the shot to be relit).
  • Expected input: A single reference video.
  • Preprocessing: Re-encode to a clean H.264 MP4 at the output frame rate (24 fps) before inference; resample the frame rate if the source differs (the reference loader does not resample temporally). Spatial resizing/cropping to the target resolution is handled automatically.
  • Alignment: The generated night video matches the reference frame-for-frame. Output frame count should satisfy frames % 8 == 1 and dimensions must be divisible by 32; the reference is sampled to the requested number of frames.
  • Mask support: Not supported.

How It Works

The reference (day) video is encoded by the VAE and supplied as in-context conditioning alongside the text prompt. The model generates a new video that keeps the reference's geometry and motion but replaces daytime lighting with night lighting. The prompt steers the style of night (e.g. moonlight, color temperature, brightness), while the reference dictates structure and movement.

Usage

🔌 ComfyUI

  1. Copy the LoRA weights into models/loras.
  2. Load the LTX-2.3-22B base model and add ltx-2.3-22b-ic-lora-day-to-night-0.9.safetensors as the LoRA.
  3. Start at strength 1.0 and adjust to taste.
  4. Use an IC-LoRA (video-to-video) workflow from the LTX-2 ComfyUI repository, which already wires the reference-video conditioning nodes. Connect your daytime clip as the reference input. Since the reference downscale factor is 1, a standard reference loader is fine.

Recommended Settings

  • LoRA strength / weight: 1.0
  • Inference steps: 30
  • Guidance scale: 3.0–4.0 (lower ≈ brighter, more preserved detail; higher ≈ darker, stronger night)
  • Resolution & frames: Trained at 768×448 (landscape) and 448×768 (portrait), 97 frames @ 24 fps (~4s). These give the best results; longer clips are possible but may drift.
  • Spatial guidance: STG mode stg_v, scale 1.0, block [29].
  • Prompting: Describe the desired night look — e.g. "A realistic nighttime scene … photorealistic moonlight, deep natural shadows. Only the lighting changes from day to night; identical composition, framing, camera movement and motion." Recommended negative prompt: daytime, bright sunlight, blue sky, overexposed, worst quality, inconsistent motion, blurry, jittery, distorted. The reference drives structure, so the prompt mainly controls lighting/brightness/color temperature.

References

Tips & Troubleshooting

  • Output too dark / crushed shadows: Lower the guidance scale (e.g. 4.0 → 3.0) and add pitch black, underexposed, crushed shadows, too dark to the negative prompt.
  • Color temperature: Steer it in the prompt — "warm tungsten interior light" vs. "cool white LED light" produce noticeably different night palettes.
  • Motion timing looks off: Make sure the reference is resampled to 24 fps before inference; the reference loader reads frames at native rate without temporal resampling.
  • Drift on long clips: For maximum fidelity, run the first ~4s (97 frames); longer references can lose consistency toward the end.

Dataset

The model was trained using a proprietary dataset of 192 motion-aligned day/night video pairs, where each pair is the identical shot rendered in daylight and at night.

Training

  • Technique: IC-LoRA (rank 32, alpha 32, dropout 0.0) on the DiT transformer (attention q/k/v/out and feed-forward projections).
  • Hyperparameters: bf16 mixed precision, AdamW, learning rate 2e-4, linear scheduler, batch size 1, gradient checkpointing, flow-matching with shifted-logit-normal timestep sampling, first-frame conditioning probability 0.2.
  • Steps: 3000 (final checkpoint recommended).
  • Resolution buckets: 768×448×97 and 448×768×97.
  • Infrastructure: LTX-2 Community Trainer.

License

See the LTX-2-community-license for full terms.

Acknowledgments