Search papers, labs, and topics across Lattice.
LeapTalk addresses the latency-quality trade-off in talking-head generation by introducing a novel framework that enables stable, real-time video generation with a single forward step. By employing a data-to-data transport formulation based on a Brownian bridge and a heterogeneous distillation framework, LeapTalk effectively reduces identity drift and enhances temporal stability. Extensive experiments show that it achieves high-fidelity video generation at up to 200 FPS, outperforming existing methods in both efficiency and stability.
LeapTalk achieves real-time talking-head generation at 200 FPS without sacrificing quality, breaking the long-standing latency-quality trade-off.
Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas real-time autoregressive approaches suffer from error accumulation and identity drift. To address this drawback, we propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. At the heart of our approach lies a single-step bridge distillation scheme. On the one hand, departing from the conventional noise-to-data paradigm, we introduce a data-to-data transport formulation based on a Brownian bridge. Anchored by a persistent reference, this strategy effectively mitigates identity drift and enhances long-term temporal stability. On the other hand, to enable smooth knowledge transfer from a pre-trained diffusion teacher to the student bridge model, we explore a heterogeneous distillation framework with an SNR-aligned time transformation 桅(蟿), which bridges the functional discrepancy between the two models. Moreover, we propose an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction. Extensive experiments demonstrate that our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS, significantly outperforming existing approaches in both efficiency and stability. Project Page: https://zhangrongxiang.github.io/leaptalk-page/