Search papers, labs, and topics across Lattice.
This paper introduces CrossMaps, a real-time semantic mapping pipeline that enhances rover navigation by constructing language-queryable maps from RGB-D data while accounting for sensor quality. By integrating multi-scale CLIP embeddings with a dual-memory architecture, CrossMaps effectively distinguishes between noisy observations and reliable semantic landmarks, promoting the latter to long-term memory for persistent navigation support. The system operates in real time on a Jetson Orin-powered UGV, enabling natural language queries to guide navigation decisions based on semantic heatmaps.
Rovers can now navigate more intelligently by querying real-time semantic maps with natural language, thanks to a novel confidence-aware mapping approach.
Rovers rely on perception to maintain spatial maps that encode both objects and sensor quality (e.g., range reliability, lighting artifacts, data density), guiding data fusion, embedding updates, and navigation under partial observability. To study these coupled perception-navigation processes, we present CrossMaps, a real-time confidence-aware open-vocabulary semantic mapping pipeline that constructs language-queryable maps from RGB-D data. Building on VLMaps-style approaches, CrossMaps integrates multi-scale CLIP embeddings with confidence-aware fusion and a dual-memory architecture consisting of Short-Term Memory (STM) and Long-Term Memory (LTM). The STM aggregates noisy visual observations using geometric, semantic, and temporal confidence cues, while confident and coherent cells are promoted to the LTM as persistent semantic landmarks. Designed for deployment with a Jetson Orin-powered UGV alongside SLAM, CrossMaps runs in real time and produces semantic heatmaps that can be queried with natural language to guide rover navigation.
