Exosfeer commited on
Commit
5391979
·
1 Parent(s): 9aebeda

Polish README for public release: add tags, model link, features section, performance stats

Browse files
Files changed (1) hide show
  1. README.md +45 -25
README.md CHANGED
@@ -7,45 +7,65 @@ sdk: gradio
7
  sdk_version: 5.23.0
8
  python_version: '3.12'
9
  app_file: app.py
10
- pinned: false
11
  license: other
12
  license_name: ltx-2-community-license-agreement
13
  license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE
14
- short_description: LTX-2.3 video+audio generation on free ZeroGPU
 
 
 
 
 
 
 
 
 
 
 
 
15
  ---
16
 
17
  # LTX-2.3 Turbo (ZeroGPU)
18
 
19
  Generate synchronized **video + audio** from text or images using
20
- [Lightricks/LTX-2.3](https://huggingface.co/Lightricks/LTX-2.3) — a 22B
21
- parameter DiT-based audio-video foundation model — running on **free ZeroGPU**
22
- hardware.
 
 
 
 
 
 
 
 
 
 
23
 
24
  ## How it works
25
 
26
- This Space uses a **vendored packages + pre-loaded models** strategy
27
- (inspired by [alexnasa/ltx-2-TURBO](https://huggingface.co/spaces/alexnasa/ltx-2-TURBO))
28
- to run the massive 22B model on ZeroGPU:
29
 
30
  1. **Startup**: Downloads model files, constructs `ModelLedger`, loads the
31
  Gemma-3 12B text encoder, and pre-loads the FP8-quantized transformer +
32
  video encoder into the pipeline cache.
33
- 2. **Text Encoding** (`@spaces.GPU`): Uses the pre-loaded text encoder to
34
- encode the prompt into video/audio context tensors, returned to CPU.
35
  3. **Video Generation** (`@spaces.GPU`): Runs the two-stage distilled
36
  denoising pipeline (8 steps low-res + 4 steps high-res with 2x spatial
37
  upscaling) using pre-encoded contexts, then decodes video and audio.
38
 
39
- ### Key optimizations for ZeroGPU
40
 
41
- - **Vendored packages**: `ltx-core` and `ltx-pipelines` are vendored directly
42
- in the repo for faster builds and customized ZeroGPU compatibility.
43
- - **FP8 quantization**: Transformer weights are cast to `float8_e4m3fn`,
44
- halving VRAM usage with minimal quality impact.
45
- - **Pre-loaded models**: Text encoder, transformer, and video encoder are
46
- loaded once at startup and kept in memory across requests.
47
- - **Distilled pipeline**: Only 8+4 denoising steps (vs 30+ for the full model),
48
- dramatically reducing inference time.
49
 
50
  ## Parameters
51
 
@@ -54,9 +74,9 @@ to run the massive 22B model on ZeroGPU:
54
  | Mode | Text to Video / Image to Video | Text to Video | |
55
  | Prompt | Free text | — | Describe scene, motion, and audio |
56
  | Resolution | 768x512, 512x512, 512x768 | 768x512 | Upscaled 2x by spatial upscaler |
57
- | Duration | 15 seconds | 2s | Shorter = more reliable on ZeroGPU |
58
- | Enhance prompt | On/Off | On | Uses Gemma to enhance the prompt |
59
- | Seed | 0–2^31 | Random | For reproducibility |
60
 
61
  ## Limitations
62
 
@@ -69,6 +89,6 @@ to run the massive 22B model on ZeroGPU:
69
 
70
  ## Credits
71
 
72
- - Model: [Lightricks/LTX-2.3](https://huggingface.co/Lightricks/LTX-2.3)
73
- - Codebase: [Lightricks/LTX-2](https://github.com/Lightricks/LTX-2)
74
- - ZeroGPU architecture inspired by [alexnasa/ltx-2-TURBO](https://huggingface.co/spaces/alexnasa/ltx-2-TURBO)
 
7
  sdk_version: 5.23.0
8
  python_version: '3.12'
9
  app_file: app.py
10
+ pinned: true
11
  license: other
12
  license_name: ltx-2-community-license-agreement
13
  license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE
14
+ short_description: Generate video + audio with LTX-2.3 (22B) on free ZeroGPU
15
+ tags:
16
+ - video-generation
17
+ - audio-generation
18
+ - text-to-video
19
+ - image-to-video
20
+ - ltx
21
+ - lightricks
22
+ - zerogpu
23
+ - fp8
24
+ - distilled
25
+ models:
26
+ - Lightricks/LTX-2.3
27
  ---
28
 
29
  # LTX-2.3 Turbo (ZeroGPU)
30
 
31
  Generate synchronized **video + audio** from text or images using
32
+ [Lightricks/LTX-2.3](https://huggingface.co/Lightricks/LTX-2.3) — a **22B
33
+ parameter** DiT-based audio-video foundation model — running entirely on
34
+ **free ZeroGPU** hardware. No paid GPU required.
35
+
36
+ **~10 seconds** to generate a 2-second video with audio at 768x512.
37
+
38
+ ## Features
39
+
40
+ - **Text to Video** — describe a scene and get video with synchronized audio
41
+ - **Image to Video** — provide a first frame and animate it with audio
42
+ - **Prompt enhancement** — Gemma-3 12B rewrites your prompt for better results
43
+ - **Multiple resolutions** — 16:9, 1:1, and 9:16 aspect ratios
44
+ - **Reproducible** — set a seed for consistent outputs
45
 
46
  ## How it works
47
 
48
+ This Space uses a **vendored packages + pre-loaded models** strategy to fit
49
+ the massive 22B model on ZeroGPU's A10G (40GB VRAM):
 
50
 
51
  1. **Startup**: Downloads model files, constructs `ModelLedger`, loads the
52
  Gemma-3 12B text encoder, and pre-loads the FP8-quantized transformer +
53
  video encoder into the pipeline cache.
54
+ 2. **Text Encoding** (`@spaces.GPU`): Encodes the prompt into video/audio
55
+ context tensors using the pre-loaded text encoder, returned to CPU.
56
  3. **Video Generation** (`@spaces.GPU`): Runs the two-stage distilled
57
  denoising pipeline (8 steps low-res + 4 steps high-res with 2x spatial
58
  upscaling) using pre-encoded contexts, then decodes video and audio.
59
 
60
+ ### Key optimizations
61
 
62
+ | Optimization | Details |
63
+ |---|---|
64
+ | **FP8 quantization** | Transformer weights cast to `float8_e4m3fn`, halving VRAM usage |
65
+ | **Distilled pipeline** | Only 8+4 denoising steps (vs 30+ for full model) |
66
+ | **Pre-loaded models** | Text encoder, transformer, video encoder loaded once at startup |
67
+ | **Two-stage upscaling** | Generates at half resolution, then upscales 2x with spatial upsampler |
68
+ | **Vendored packages** | `ltx-core` and `ltx-pipelines` bundled for fast builds |
 
69
 
70
  ## Parameters
71
 
 
74
  | Mode | Text to Video / Image to Video | Text to Video | |
75
  | Prompt | Free text | — | Describe scene, motion, and audio |
76
  | Resolution | 768x512, 512x512, 512x768 | 768x512 | Upscaled 2x by spatial upscaler |
77
+ | Duration | 1-5 seconds | 2s | Shorter = more reliable on ZeroGPU |
78
+ | Enhance prompt | On/Off | On | Gemma-3 rewrites prompt for better results |
79
+ | Seed | 0-2B | Random | For reproducibility |
80
 
81
  ## Limitations
82
 
 
89
 
90
  ## Credits
91
 
92
+ - **Model**: [Lightricks/LTX-2.3](https://huggingface.co/Lightricks/LTX-2.3) (22B parameters)
93
+ - **Codebase**: [Lightricks/LTX-2](https://github.com/Lightricks/LTX-2)
94
+ - **ZeroGPU architecture** inspired by [alexnasa/ltx-2-TURBO](https://huggingface.co/spaces/alexnasa/ltx-2-TURBO)