Search papers, labs, and topics across Lattice.
The paper introduces a new task, Holistic Affordance Grounding in 360° Indoor Environments, to enable embodied agents to better understand affordances in panoramic views. To address challenges like geometric distortions and semantic dispersion, they propose PanoAffordanceNet, an end-to-end framework with a Distortion-Aware Spectral Modulator (DASM) and Omni-Spherical Densification Head (OSDH). Experiments on their new 360-AGD dataset show PanoAffordanceNet significantly outperforms existing methods, establishing a new baseline for scene-level perception.
Overcome the limitations of object-centric affordance grounding with a new model that understands affordances holistically in 360° indoor environments, even with geometric distortions.
Global perception is essential for embodied agents in 360° spaces, yet current affordance grounding remains largely object-centric and restricted to perspective views. To bridge this gap, we introduce a novel task: Holistic Affordance Grounding in 360° Indoor Environments. This task faces unique challenges, including severe geometric distortions from Equirectangular Projection (ERP), semantic dispersion, and cross-scale alignment difficulties. We propose PanoAffordanceNet, an end-to-end framework featuring a Distortion-Aware Spectral Modulator (DASM) for latitude-dependent calibration and an Omni-Spherical Densification Head (OSDH) to restore topological continuity from sparse activations. By integrating multi-level constraints comprising pixel-wise, distributional, and region-text contrastive objectives, our framework effectively suppresses semantic drift under low supervision. Furthermore, we construct 360-AGD, the first high-quality panoramic affordance grounding dataset. Extensive experiments demonstrate that PanoAffordanceNet significantly outperforms existing methods, establishing a solid baseline for scene-level perception in embodied intelligence. The source code and benchmark dataset will be made publicly available at https://github.com/GL-ZHU925/PanoAffordanceNet.
