Search papers, labs, and topics across Lattice.
DXPR unifies monocular camera queries and LiDAR scans into a shared depth image representation, enabling a single vision foundation model backbone to extract modality-invariant global descriptors without separate, modality-specific encoders. To ensure reliable metric learning under geometric misalignment, the framework introduces a geometry-aware overlap miner that computes pixel-level overlap via forward-warping to adaptively modulate positive margins within a multi-similarity loss. On the KITTI and Boreas benchmarks, the approach achieves near-perfect Recall@1 on KITTI and consistently outperforms prior cross-modal baselines like RangeBEV across severe seasonal, weather, and day-night shifts.
Monocular camera localization within dense LiDAR maps no longer requires complex dual-encoder architectures: converting both inputs to unified depth representations lets a single vision foundation model outperform specialized cross-modal baselines under extreme environmental shifts.
We present DXPR, a depth-based cross-modal place recognition (CMPR) framework that uses vision foundation models (VFMs) to match monocular camera queries against a LiDAR map without modality-specific encoders. This enables robots and autonomous vehicles to robustly localize using only cameras within pre-built LiDAR maps, even under severe seasonal, weather, and illumination changes. The key idea is to convert both camera images and LiDAR scans into a unified depth image representation so that a single VFM backbone with an aggregation head can learn modality-invariant global descriptors. To make pairwise metric learning faithful to scene geometry, we introduce a geometry-aware overlap miner: after cross-modal scale alignment of camera and LiDAR depth, we forward-warp measurements between views to compute a pixel-level overlap score. This score relabels ambiguous pairs and adaptively modulates the positive margin in a multi-similarity loss to avoid overfitting on weakly overlapping views. Extensive experiments on KITTI odometry and Boreas demonstrate strong performance and robustness across seasons, weather, and day/night. On KITTI, DXPR achieves near-perfect Recall@1 on most sequences and outperforms prior CMPR baselines. On Boreas, DXPR achieves intra-sequence performance on par with a strong single-modal baseline (DINOv2-SALAD), while showing clear improvements in the more challenging inter-sequence setting. Compared with RangeBEV, our method consistently performs better in both intra- and inter-sequence evaluations, demonstrating robustness under diverse seasonal and illumination changes.
