Search papers, labs, and topics across Lattice.
This paper introduces DeSeG, a hierarchical framework designed to decouple semantic intent from geometric constraints in human-scene interactions (HSI). By employing a Residual Semantic Planner for fine-grained semantic control and a physics regularized diffusion executor to enforce collision-aware motion, the framework addresses the issues of semantic-geometric entanglement and physical hallucinations. Experimental results on the Lingo dataset show that DeSeG reduces mean scene penetration by 47% and enhances semantic alignment by 29% compared to state-of-the-art baselines.
Decoupling semantic intent from geometric constraints leads to a 47% reduction in physical hallucinations during human-scene interactions.
Synthesizing physically plausible human-scene interactions (HSI) remains a critical challenge in computer vision and the development of human avatars. Although recent generative models enable diverse motion synthesis, they suffer from an inductive bias referred to as semantic-geometric entanglement. Because spatial constraints often strongly correlate with specific actions in training data, monolithic models will learn the shortcut bias, aggressively overriding the semantic intent when faced with strict geometric cues. Furthermore, this entanglement exacerbates physical hallucinations, such as body-scene penetrations. To address these limitations, we propose DeSeG, a hierarchical framework that explicitly decouples semantic intent from geometric constraints. First, we introduce a Residual Semantic Planner that encodes textual instructions and canonicalized goal voxels into a compact latent space, enabling fine-grained semantic control independent of spatial trajectories. Second, we propose a physics regularized diffusion executor that incorporates differentiable repulsive potential fields directly into the diffusion objective, enforcing collision-aware motion generation. Extensive experiments on the Lingo dataset demonstrate that DeSeG achieves state-of-the-art performance, reducing mean scene penetration by 47% and improving semantic alignment by 29% over the SOTA baselines.
