README.md

metadata

tags:
  - ahead-of-time
  - pytorch
  - ltx-2.3
  - zerogpu
library_name: diffusers
base_model: diffusers/LTX-2.3-Diffusers

LTX-2.3 Transformer — AOTI build (in-context, STG-capable)

Ahead-of-time precompiled transformer_blocks of LTX2VideoTransformer3DModel for ZeroGPU (sm120 / cu130). Graph only (no weights) → works with base and distilled LTX-2.3 and any fused LoRA. Dynamic over video/audio token counts.

This build compiles the perturbation (STG) path as always-on tensor math (torch.lerp(value, hidden_states, perturbation_mask)), so ONE graph serves both spatio-temporal-guidance (STG) and non-STG: lerp(·, ·, ones) is a no-op, and the real mask blends at the STG block. Use this for the base-model demos that keep STG (stg_scale>0, default spatio_temporal_guidance_blocks=[28]) — e.g. beard-removal, day-to-night, reference-sheet. (Non-STG demos can use the plain Group A repo.)

Use it (ZeroGPU) — load at the root module level + a small STG wrapper

import spaces, torch
from diffusers import LTX2InContextPipeline

pipe = LTX2InContextPipeline.from_pretrained(
    "diffusers/LTX-2.3-Diffusers", torch_dtype=torch.bfloat16).to("cuda")
pipe.load_lora_weights(my_lora_state_dict, adapter_name="x")
pipe.fuse_lora(lora_scale=1.0); pipe.unload_lora_weights()
spaces.aoti_load(module=pipe.transformer, repo_id="ltx-community/LTX-2.3-Transformer-GroupC-STG-sm120-cu130-rb3")

# the compiled graph always runs the perturbation lerp, so feed a no-op ones mask when the
# transformer passes None (non-STG blocks / main pass); the STG pass still passes the real
# mask to block 28. Also force all_perturbed=False (the python skip-attention shortcut is gone).
for _blk in pipe.transformer.transformer_blocks:
    _c = _blk.forward
    def _fwd(*a, _c=_c, **kw):
        if kw.get("perturbation_mask", None) is None:
            _h = kw["hidden_states"]
            kw["perturbation_mask"] = torch.ones((_h.shape[0],1,1), device=_h.device, dtype=_h.dtype)
        kw["all_perturbed"] = False
        return _c(*a, **kw)
    _blk.forward = _fwd

@spaces.GPU
def generate(*args, **kwargs):
    return pipe(*args, **kwargs)

Public repo (graph only) → no token. Built with the bundled job.py (env LTX_STG=1).