Search papers, labs, and topics across Lattice.
This paper introduces EAGOR, a geometry-aware framework designed for embodied directional reasoning using 360-degree camera inputs, addressing the limitations of existing Vision Language Models that rely on 2D equirectangular projections. By formulating directional reasoning as recursive Bayesian estimation on the spherical manifold, EAGOR maintains a continuous belief over target directions that adapts to agent motion without requiring training of the underlying VLMs. The results demonstrate significant performance improvements, with EAGOR achieving average relative gains of +34.4% and +45.6% on benchmark datasets, alongside enhanced navigation metrics in real-world robotic applications.
EAGOR revolutionizes embodied reasoning by directly leveraging the spherical nature of 360-degree observations, leading to remarkable gains in navigation accuracy and efficiency.
Omni-directional (360{\deg}) cameras provide embodied agents with a holistic view of their surroundings, making them suited for directional reasoning in tasks such as navigation and object search. Existing Vision Language Models (VLMs) project 360{\deg} observations to 2D equirectangular projection (ERP) images and process them using architectures designed for perspective images. However, they ignore the spherical nature of 360{\deg} observations, where each pixel represents a viewing direction relative to the agent. Consequently, their direction estimates often become inconsistent under camera view transformations caused by agent motion. This limitation is particularly critical for map-free navigation, where the agent must continuously estimate the target direction in its egocentric frame. We propose EAGOR, a training-free, geometry-aware framework for embodied 360{\deg} directional reasoning. Instead of predicting target directions as ERP image coordinates, EAGOR formulates directional reasoning as recursive Bayesian estimation directly on the sphere. It maintains a continuous belief over target directions and propagates it equivariantly under agent motion without training the backbone VLMs. To achieve this, we introduce the Spherical Harmonic Belief Field (SH-BF), whose spherical harmonic representation provides a globally defined, rotation-aware basis for directional estimation on the spherical manifold. This formulation eliminates ERP seam discontinuities, latitude distortions, and interpolation errors. We evaluate EAGOR on two benchmark datasets and real-world experiments with a legged robot across directional reasoning tasks. EAGOR consistently outperforms existing methods, achieving average relative gains of +34.4% and +45.6% on HOS and OSR-Bench, respectively, while improving navigation success by +14.6%, reducing step count by 17.7%, and lowering mean angular error by 24.5%.
