Search papers, labs, and topics across Lattice.
SurroundNEXO introduces a novel framework for multi-camera depth prediction in autonomous driving that leverages ego-centric geometry instead of traditional dense visual correspondences. By utilizing Ego-Ray Positional Encoding and sparse LiDAR measurements, the method enhances spatial consistency and metric-scale depth accuracy across weakly overlapping camera views. The results show a significant reduction in single-view error by 33.2% and improvements in cross-view consistency and metric reconstruction quality by 10.5% and 25.6%, respectively, compared to state-of-the-art approaches.
Bridging the gap in multi-camera depth prediction, SurroundNEXO achieves a 33.2% reduction in single-view error by rethinking how we leverage ego-centric geometry.
Modern autonomous driving depends on accurate metric 3D understanding for perception, reconstruction, and planning, which in turn requires reliable multi-camera depth prediction. However, the outward-facing nature of vehicle-mounted surround-view camera rigs inherently limits visual overlap across views, challenging the correspondence-based assumptions that underpin conventional multi-view geometry. To bridge this gap, we present SurroundNEXO, named after the Spanish word nexo for a geometric link, a low-overlap multi-camera metric depth framework that grounds cross-view reasoning in ego-centric geometry rather than dense visual correspondences. Instead of directly enforcing early global fusion, SurroundNEXO first assigns image tokens globally comparable ego-frame viewing directions through Ego-Ray Positional Encoding, then uses sparse LiDAR measurements as metric anchors to propagate absolute scale cues, and finally expands feature interaction progressively from view-local modeling to decomposed spatio-temporal reasoning and global integration. This design enables metric-scale depth prediction with improved spatial consistency across weakly overlapping cameras. Across low-overlap autonomous driving benchmarks, including NuScenes, Waymo and DDAD, SurroundNEXO reduces single-view error by 33.2%, improves cross-view consistency by 10.5%, and enhances metric reconstruction quality by 25.6% compared with SOTA methods. It further remains robust under extremely sparse depth prompts and exhibits strong zero-shot generalization to unseen camera layouts.
