Search papers, labs, and topics across Lattice.
DiscoForcing is introduced as a streaming audio-driven diffusion framework for real-time character control. It combines a causal music encoder with a diffusion-forcing sequence model trained under heterogeneous noise levels to handle non-stationary audio. The system employs a hybrid temporal schedule and history-guided streaming sampler to balance responsiveness and long-horizon consistency, achieving superior stability and audio-motion alignment compared to existing methods while maintaining real-time performance.
Streaming audio-driven character animation can finally handle abrupt tempo changes and user edits in real-time, thanks to a diffusion-forcing approach that prioritizes both responsiveness and long-term coherence.
We study real-time audio-responsive character control as a deployment-faithful problem: strictly causal, bounded-latency streaming that must generate coherent full-body motion at interactive frame rates while the audio condition can change abruptly, including tempo shifts, drops, or user edits. Prior music-to-motion systems are largely optimized for offline generation with global context, and degrade in streaming rollouts where conditioning history becomes stale or unreliable. We introduce DiscoForcing, a streaming audio-driven diffusion framework that combines a causal music encoder that captures rhythmic structure and phase dynamics with a diffusion-forcing sequence model trained under heterogeneous noise levels across the temporal horizon. Building on this, we design a hybrid temporal schedule and a history-guided streaming sampler to explicitly trade off responsiveness against long-horizon consistency under non-stationary audio. Implemented in an end-to-end real-time interactive system with online avatar playback and humanoid deployment workflows, DiscoForcing delivers more stable long-horizon rollouts and sharper audio-motion alignment than prior baselines under matched causality and latency constraints while maintaining real-time throughput.
