Search papers, labs, and topics across Lattice.
This paper introduces OpenReLoc, a novel camera relocalization system that organizes rich object information into structured map representations to enhance indoor visual relocalization. By integrating open-vocabulary semantic knowledge through a multi-modal mechanism and employing object-oriented reference frames, the system effectively improves 2D-3D object matching and pose estimation. Experimental results show that OpenReLoc significantly outperforms existing methods in relocalization recall and accuracy across multiple datasets, highlighting its potential for spatial and embodied AI applications.
OpenReLoc achieves unprecedented relocalization accuracy by leveraging open-vocabulary semantics and structured object representations, setting a new standard for indoor navigation systems.
Indoor visual relocalization plays a critical role in emerging spatial and embodied AI applications. However, prior research was predominantly devoted to low-level vision schemes, struggling to perceive scene semantics and compositions, which limits both interpretability and applicability. In this paper, we explore the issue of how to organize rich object information in a scene, including semantics, layout, and geometry, into a structured map representation, thereby utilizing object units exclusively to drive the camera relocalization task. To this end, we propose OpenReLoc, a camera relocalization system designed to provide scene understanding and accurate pose estimation capabilities. Leveraging recent foundation models, we first introduce a multi-modal mechanism to integrate open-vocabulary semantic knowledge for effective 2D-3D object matching. Additionally, we design object-oriented reference frames as position priors, paired with a reference frame selection strategy based on the Distance-IoU (DIOU), enabling extension to scalable scenes. Moreover, to ensure stable and accurate pose optimization, we also propose a dual-path 2D Iterative Closest Pixel loss guided by object shape. Experimental results demonstrate that OpenReLoc achieves superior relocalization recall and accuracy across various datasets. Our source code will be released upon acceptance.
