Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
EvolvingAvatar implements a causal 3D head generation framework that applies self-supervised test-time training to continually adapt to streaming user video and dyadic audio without target motion labels. By decoupling adaptation into persistent fast weights that accumulate conversational history and transient weights for immediate jaw synchronization, the system continuously refines speaking and listening dynamics. Evaluated on the new 455.95-hour InterHead-Bench, the model progressively improves as interactions unfold, reducing out-of-distribution user-avatar expression mismatch by up to 11.1%.
Conversational 3D heads no longer need static weights: turning live dyadic audiovisual streams into an online self-supervised training signal makes avatar motion up to 11.1% more coherent as dialogues unfold.
Interactive 3D head generation requires coordinated speaking and listening motion that responds to an evolving conversation. Existing generators use incoming observations as context but keep their parameters fixed, leaving conversational patterns unused as a learning signal. We introduce EvolvingAvatar, a causal generator that uses test-time training to adapt to user face video and dyadic audio during interaction. Its dyadic context prediction objective provides a self-supervised learning signal from audiovisual context without target motion labels at test time. Persistent fast weights accumulate these updates within each conversation to guide motion generation, while transient jaw adaptation responds to current audiovisual context. Predicted speech activity controls how persistent adaptation guides motion. We also introduce InterHead-Bench, a unified 455.95-hour benchmark built from single-view and dual-view conversation videos. Experiments show improved conversational motion statistics over strong baselines. On the hardest out-of-distribution split, generation improves as conversations unfold, reducing mismatch with recorded user-avatar expression statistics by up to 11.1% from the first interval.