Search papers, labs, and topics across Lattice.
The paper introduces ReMoGen, a modular learning framework for generating real-time human motion reactions conditioned on multi-source cues like other agents' actions and scene geometry. It addresses the challenge of limited and fragmented interaction data by leveraging a universal motion prior adapted to specific interaction domains via independently trained Meta-Interaction modules. ReMoGen achieves real-time performance and coherence through segment-level generation with a lightweight frame-wise refinement module, outperforming existing methods in human-human, human-scene, and mixed-modality interaction settings.
Achieve realistic and responsive avatar interactions without expensive full-sequence inference, thanks to a novel modular framework that adapts pre-trained motion priors to diverse interaction scenarios.
Human behaviors in real-world environments are inherently interactive, with an individual's motion shaped by surrounding agents and the scene. Such capabilities are essential for applications in virtual avatars, interactive animation, and human-robot collaboration. We target real-time human interaction-to-reaction generation, which generates the ego's future motion from dynamic multi-source cues, including others' actions, scene geometry, and optional high-level semantic inputs. This task is fundamentally challenging due to (i) limited and fragmented interaction data distributed across heterogeneous single-person, human-human, and human-scene domains, and (ii) the need to produce low-latency yet high-fidelity motion responses during continuous online interaction. To address these challenges, we propose ReMoGen (Reaction Motion Generation), a modular learning framework for real-time interaction-to-reaction generation. ReMoGen leverages a universal motion prior learned from large-scale single-person motion datasets and adapts it to target interaction domains through independently trained Meta-Interaction modules, enabling robust generalization under data-scarce and heterogeneous supervision. To support responsive online interaction, ReMoGen performs segment-level generation together with a lightweight Frame-wise Segment Refinement module that incorporates newly observed cues at the frame level, improving both responsiveness and temporal coherence without expensive full-sequence inference. Extensive experiments across human-human, human-scene, and mixed-modality interaction settings show that ReMoGen produces high-quality, coherent, and responsive reactions, while generalizing effectively across diverse interaction scenarios.
