README.md

metadata

title: Krea Realtime Video
emoji: 🎬
colorFrom: blue
colorTo: purple
sdk: gradio
app_file: app.py
hardware: zerogpu
python_version: 3.10.13
short_description: Realtime T2V demo via diffusers ModularPipeline

Krea Realtime Video 14B

This Space implements real local inference for krea/krea-realtime-video using the official 🧨 Diffusers ModularPipeline path from the model card.

What it does

• Text-to-video: enter a prompt, choose how many blocks (≈ frames) to generate, and hit Generate Video.
• The model is a 14B autoregressive video diffusion model distilled from Wan 2.1 T2V 14B.

How to try it

1. Open the Generate Video tab.
2. Type a prompt (e.g. "a cat sitting on a boat").
3. Adjust Number of Blocks (1–9) and Inference Steps per Block (1–20).
   • Lower values = faster and cheaper first test.
4. Click Generate Video.

Example inputs

• a cat sitting on a boat (3 blocks, 4 steps, seed 42)
• a futuristic city at sunset (3 blocks, 4 steps, seed 123)
• a panda playing guitar in a forest (2 blocks, 4 steps, seed 7)

Limitations & known risks

• VRAM: the 14B transformer (28 GB in bf16) plus the UMT5-XXL text encoder (12–14 GB) and VAE push the full pipeline well above 48 GB. Standard HF Space GPU tiers are unlikely to fit the model without aggressive quantization or >96 GB of VRAM.
• Hardware recommendation: this demo should be run on a Space with a very large GPU (e.g. xlarge ZeroGPU tier or a dedicated high-memory GPU).
• If the pipeline fails to load, the /health endpoint and the UI surface the concrete load error (OOM, CUDA init, missing import, etc.).
• The model card’s recommended optimizations (torch.compile, SageAttention, FP8 quantization with torchao) are not enabled here because:
  • torch.compile is not supported on ZeroGPU.
  • flash-attn / kernels / torchao are native-build dependencies that frequently fail in HF Space build environments.
• We therefore rely on the SDPA fallback and vanilla bf16/fp16 precision.

Architecture

• app.py loads ModularPipeline.from_pretrained(..., trust_remote_code=True) at module level so ZeroGPU can pack resident tensors.
• Generation is guarded by @spaces.GPU and uses iterative PipelineState blocks exactly as the model card demonstrates.
• Health (api_name="health") is instant and never touches weights.

Dependencies

• diffusers (main branch from GitHub)
• torch, transformers, accelerate, safetensors, spaces, gradio, tqdm