Search papers, labs, and topics across Lattice.
This paper introduces PSG-Nav, a novel approach to open-vocabulary navigation that leverages a 3D Probabilistic Scene Graph to manage perception uncertainty through semantic categorical distributions. By employing Multiverse Decision making, the method samples multiple likely world settings to evaluate navigation landmarks, enhancing decision-making beyond local optima. The proposed Evidential Experience Calibrator further improves performance by adapting to past experiences, resulting in state-of-the-art success rates on benchmark datasets MP3D, HM3D, and HSSD.
PSG-Nav achieves unprecedented navigation success rates by integrating probabilistic reasoning with lifelong learning, effectively tackling perception uncertainty in complex environments.
Open-vocabulary navigation requires embodied agents to manage significant perception uncertainty stemming from semantic ambiguity and model errors. However, most existing works settle for local optimal deterministic approaches, depriving complex navigation decision-making over multiple composite possibilities that are critical for globally better solutions. In this paper, we propose Probabilistic Scene Graph Navigation (PSG-Nav), which constructs a 3D Probabilistic Scene Graph that uses full semantic categorical distributions to account for perception uncertainty. To efficiently use the local distributions to compose and reason about the optimal navigation landmarks, we propose Multiverse Decision to sample multiple most likely world settings from the joint distribution, and evaluate navigation landmarks based on the compatibility between landmarks and multiverses. To mitigate false positives due to epistemic uncertainty in open-vocabulary navigation, we introduce the Evidential Experience Calibrator, which enables online lifelong adaptation by cross-validating detections against memories of past successes and failures. Extensive experiments on widely-used benchmarks MP3D, HM3D, and HSSD demonstrate that PSG-Nav establishes new state-of-the-art results, achieving Success Rates of 66.1%, 44.8%, and 67.9%, respectively. Code is available at: https://psg-nav.github.io/
