Search papers, labs, and topics across Lattice.
This paper introduces InteractGesture, a model-agnostic method that enhances spatial controllability in co-speech gesture generation by employing Progressive Chunk Guidance to manage chunk-wise dependencies during streaming. By allowing spatial control gradients to propagate backward across chunk boundaries, the method addresses the limitations of existing models that freeze prior chunks, leading to boundary inconsistencies. Experiments demonstrate that InteractGesture significantly improves multi-joint spatial control while maintaining high gesture quality, showcasing its versatility for various applications in gesture generation.
Spatial control in co-speech gesture generation can now be fine-tuned in real-time, overcoming the limitations of traditional models that freeze prior motion chunks.
Co-speech gesture generation has made significant progress toward realistic full-body motion from speaker audio, yet existing models lack fine-grained spatial controllability of individual joints. To address this, we introduce \emph{InteractGesture}, a model-agnostic, inference-time method for spatially controllable gesture generation. \emph{InteractGesture} guides target latent estimates of a diffusion sampler through a differentiable RVQ-VAE decoder, backpropagating spatial control gradients to adjust motion latents during sampling. A primary challenge in streaming co-speech generation is chunk-wise dependency: standard sequential inference freezes prior chunks, preventing spatial constraints in future chunks from adjusting preceding trajectories and causing boundary inconsistencies. To overcome this limitation, we propose \emph{Progressive Chunk Guidance}, a chunk-window strategy that maintains an active set of editable chunk latents with staggered delays, enabling spatial constraints to propagate gradients backward across chunk boundaries during streaming generation. Experiments on the BEAT2 dataset show that \emph{InteractGesture} improves multi-joint spatial control while preserving overall gesture quality. Furthermore, our approach supports diverse applications, including sparse joint positioning, dense joint trajectory control, and directional pointing. Our project page is available at https://exitudio.github.io/interactgesture-page .
