LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation
Abstract
Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas real-time autoregressive approaches suffer from error accumulation and identity drift. To address this drawback, we propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. At the heart of our approach lies a single-step bridge distillation scheme. On the one hand, departing from the conventional noise-to-data paradigm, we introduce a data-to-data transport formulation based on a Brownian bridge. Anchored by a persistent reference, this strategy effectively mitigates identity drift and enhances long-term temporal stability. On the other hand, to enable smooth knowledge transfer from a pre-trained diffusion teacher to the student bridge model, we explore a heterogeneous distillation framework with an SNR-aligned time transformation Φ(τ), which bridges the functional discrepancy between the two models. Moreover, we propose an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction. Extensive experiments demonstrate that our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS, significantly outperforming existing approaches in both efficiency and stability. Project Page: https://zhangrongxiang.github.io/leaptalk-page/
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars (2026)
- OPSD-V: On-Policy Self-Distillation for Post-Training Few-Step Autoregressive Video Generators (2026)
- Lip Forcing: Few-Step Autoregressive Diffusion for Real-time Lip Synchronization (2026)
- FreqForcing: Autoregressive Long Video Generation via Spectral Self-Anchoring (2026)
- Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memory (2026)
- Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions (2026)
- Ms. Forcing: Efficient Streaming Video Generation with Multi-Scale Patchification and Attention (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2608.00079 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 1
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper