Search papers, labs, and topics across Lattice.
This paper introduces Embodied Scene Rearrangement Planning (ESRP), a challenging task where embodied agents must rearrange furniture in 3D environments using only egocentric views and a top-down target layout, without access to global state information. The ESRP task incorporates mutual object occlusions, simulating real-world constraints that complicate long-horizon planning and alignment of partial observations. Through the ESRP-Bench benchmark, which includes over 5,400 scene pairs and 8,200 objects, the authors reveal that existing methods struggle with task efficiency, marking ESRP as a significant challenge for future research in embodied agents.
Current methods falter in efficiently rearranging scenes with occlusions, exposing a critical gap in embodied agent capabilities.
This paper introduces Embodied Scene Rearrangement Planning (ESRP), a novel task requiring embodied agents to rearrange furniture in 3D scenes to match a target configuration using only egocentric observations and a top-down target layout. Unlike prior rearrangement tasks, ESRP precludes global state access and introduces mutual object occlusions, reflecting the practical constraints of real-world robotic deployment. These factors make aligning partial egocentric observations with the global target layout particularly challenging for long-horizon planning. To facilitate research, we present ESRP-Bench, a comprehensive benchmark built on OmniGibson featuring over 5,400 scene pairs and 8,200 objects. We define three multi-level metrics to evaluate rearrangement quality and provide four baselines: a hierarchical task-and-motion planning method, a vision-language-model-based method, and two learning-based approaches (IL and RL). Experimental results demonstrate that current methods struggle to complete the task efficiently, highlighting ESRP as a challenging frontier for embodied agents in scene understanding and long-horizon task planning. This work serves as a stepping stone toward deploying intelligent agents in real-world scenarios. Project page: https://pie-lab.cn/ESRP/.
