joeygambino/joyai-echo-ltx23-echoVid-ltxAud-surgical-int8

JoyAI-Echo x LTX-2.3 surgical merge — INT8 ConvRot (stock ComfyUI)

Talking-heads note (2026-07-25): same weights, same behaviour as the main repo — at conversational word rates in plain core-node workflows, stock ltx-2.3-22b-distilled-1.1 articulates mouths ~2x harder than this merge. See the main card for numbers and when each model wins.

Nodes, workflow, issues and releases: https://github.com/jlucasmcrell/joyai-echo-multishot-patch
All models: https://huggingface.co/joeygambino

A word on expectations: the multishot workflow this feeds is powerful but not one-click - first renders usually take some tuning to your VRAM, system RAM and model build. The workflow repo's INSTRUCTIONS.md covers setup and known failure modes, and I am happy to troubleshoot: open a discussion and I will answer.

The surgical merge (JoyAI-Echo's video/conditioning branch + LTX-2.3-distilled-1.1's audio branch) as a stock-ComfyUI-native INT8 checkpoint — int8_tensorwise format with ConvRot rotation, built with Comfy-Org's own comfy-quants exporter.

Built by request. Loads with the standard loader in ComfyUI v0.27.0 or newer — no custom nodes — on any GPU with SM >= 7.5 (RTX 20-series up). INT8 compute typically runs 1.5-2x faster than bf16 on 30-series cards, and the whole file is 28 GB versus 43 GB bf16.

Speed: turn on fp8_scaled_mm

If your renders crawl in the JoyAI-Echo pack, this is almost always why. It is off by default.

fp8_scaled_mm stores the DiT linears as float8_e4m3fn and computes the matmuls natively via torch._scaled_mm on RTX 40/50-series — unlike fp8_transformer there is no per-layer upcast tax.

source file in model_file	fp8_scaled_mm	result
bf16	ON	linears downcast at load, native fp8 compute — ~22 GB resident, sequential_offload can go OFF
INT8 ConvRot	ON	reconstructs to bf16, then downcasts — identical to the bf16 row, at ~60 % of the download
bf16 or INT8	off	runs in bf16 — ~40 GB staged, streams over PCIe on a 32 GB card
fp8	off	upcasts to bf16 at load — saves download size only, not memory or time
fp8	ON	hard error. The toggle downcasts from bf16 itself; an fp8 file would load norms and tables as fp8 too and crash the denoise pipeline

The short version: feed it bf16 or INT8 and turn the toggle on. The fp8 file is for running without the toggles.

(Under stock ComfyUI rather than the pack, the INT8 build instead runs on the native W8A8 loader — a different path with its own speed profile.)

Render at 24 fps — voices and accents (2026-07-30)

LTX-2.3's joint audio-video prior is 24 fps-native, and render fps turns out to be a hidden accent dial: at 25 fps the same prompt and seed render non-rhotic southern British, at 30 fps broad Australian — and any off-24 fps setting overrides accent wording in the prompt entirely (a few conditioning tokens cannot outvote a geometric signal present in every attention op at every step). Verified by A/B on identical configs: 24 fps = rhotic General American at 5/5 reviewer confidence, same text and seed.

Keep video_fps at 24 and state the accent in the positive prompt ("in a casual American accent"). Flip side: if you want an authentic British or Australian voice, rendering that scene at 25/30 fps is more consistent than any accent wording.

Demo

Low-light stress test - single lamp, smooth falloff into deep shadow, close-up skin, spoken dialogue. Rendered with this INT8 ConvRot file in stock ComfyUI 0.28. AI-generated video and audio.

What's inside

• Official Lightricks layer recipe, layer-for-layer: transformer blocks 2-45 quantized (1,496 Linears), blocks 0/1/46/47 and both embeddings connectors kept bf16 — the same selection as the official LTX-2.3 fp8/nvfp4 releases.

• ConvRot on every quantized layer (regular Hadamard rotation, group 256): rotates weights before INT8 encoding to suppress the row-wise outliers DiT architectures produce. Large quality gain over plain row-wise INT8.

• Two files, pick one. ..._int8_convrot.safetensors (27.2 GB) is the full single-file bundle: video VAE, audio VAE, vocoder, and the four text_embedding_projection.* tensors are all inside, copied verbatim in bf16 - load with CheckpointLoaderSimple, no companion downloads. ..._int8_convrot_transformer_only.safetensors (25.5 GB) is the DiT alone for people who prefer pairing it with the original full-precision LTX-2.3 VAE and their own text-encoder setup - it deliberately KEEPS the four tiny text_embedding_projection.* tensors (conversions that drop them cause the robotic-voice / prompt-reading / new-face-every-shot failure class).

• Every quantized layer carries the comfy_quant marker stock ComfyUI keys on:

  {"format": "int8_tensorwise", "convrot": true, "convrot_groupsize": 256}

Which format should you use?

you run	use	why
Stock ComfyUI >= 0.27, standard LTX-2.3 workflows	this file	native loader, INT8 speed, one file
The JoyAI-Echo multishot node pack (memory bank, one character across shots)	this file works there too (pack v1.5.2+)	the pack reconstructs the INT8 weights to bf16 at load (dequant + ConvRot un-rotation, ~0.95% from the bf16 source). Loading needs the same system RAM as the bf16 build - this file saves download size, not memory
Maximum fidelity on a 24 GB+ card	Q8_0 GGUF	weight-only quantization, compute in bf16, measured ~0.6% from bf16

Honest fidelity note: this is a W8A8-class path — weights are INT8 (ConvRot-rotated) and activations are quantized at runtime. That trades some fidelity for speed. The Q8_0 GGUF dequantizes to bf16 compute and is the higher-fidelity option; this file is the faster and more compatible one.

Verification

Exported with comfy-quants export-model-int8-tensorwise --config configs/ltx2_int8_tensorwise.yaml --convrot from the clean bf16 merge. Format-level checks passed: 1,496 quantized layers matching the official recipe count, per-output-channel F32 scales, markers on every quantized layer, first/last two blocks bf16.

Render-tested (stock ComfyUI 0.28.0, LTX-2.3 t2v+audio template, RTX 5090): a deliberate low-light stress clip — single warm lamp, smooth falloff into deep shadow, close-up skin, spoken dialogue. Result: no banding or shadow blocking in the gradients, natural skin texture with no quantization shimmer, and the speech came through clean with tight lip sync — a blind review found no quantization noise on the voice. If you hit anything odd in your workflows, open a discussion.

License

LTX-2 Community License. The JoyAI-Echo component is research / non-commercial only, and the stricter term governs your outputs. AI-generated content must be disclosed as such. Not affiliated with Lightricks, JD, RealRebelAI, or Comfy-Org.

Support

Everything here is free and stays free. If it saved you time, you can sponsor me on GitHub, buy me a coffee, or support me on Liberapay.