Search papers, labs, and topics across Lattice.
This paper introduces Test-Time Self-Adaptive Conditioning (TT-SAC), a parameter-free inference framework that refines conditioning representations in audio-driven talking-head generators by feeding the generator's outputs back into its encoder. This feedback loop creates a refined conditioning representation that adapts to the temporal dynamics of the synthesized sequence, mitigating issues like identity drift and temporal inconsistency. Experiments demonstrate that TT-SAC improves lip-sync accuracy, temporal coherence, and identity preservation across various state-of-the-art talking-head generators.
Stop identity drift in your talking-head generators without retraining: a simple feedback loop between the generator and encoder stabilizes identity and motion.
Audio-driven talking-head generation has achieved remarkable progress with recent models such as AniTalker, FLOAT, and Sonic. Despite their success, most existing approaches rely on a single static reference image to condition the entire video generation process at inference stage. This static conditioning paradigm often creates a mismatch between fixed identity features and dynamically evolving facial motion, leading to identity drift, temporal inconsistency, and degraded perceptual quality. We introduce Test-Time Self-Adaptive Conditioning (TT-SAC), a parameter-free inference framework that enables pretrained talking-head generators to adapt their conditioning representations during inference without retraining, gradient updates, or additional supervision. Instead of treating the reference portrait as immutable, TT-SAC composes the generator with its encoder in a feedback loop: the generator's own outputs are re-encoded to construct a refined conditioning representation that better aligns with the temporal dynamics of the synthesized sequence. A single adaptation step approximates a self-consistent equilibrium of the generative process, stabilizing identity and motion across time. We further provide theoretical analysis showing that test-time conditioning adaptation reduces feature variance and improves generative stability under mild Lipschitz assumptions, while exhibiting a principled bias-variance tradeoff that governs the optimal strength of adaptation. Extensive experiments on state-of-the-art talking-head generators and benchmark datasets demonstrate consistent improvements in lip-sync accuracy, temporal coherence, identity preservation, and perceptual fidelity. TT-SAC offers a model-agnostic and training-free strategy for enhancing generative video models, establishing test-time conditioning adaptation as an effective mechanism for stabilizing audio-driven portrait animation.
