Search papers, labs, and topics across Lattice.
This paper introduces Map-Det3D, an online multi-view 3D object detection model that directly predicts 3D bounding boxes from RGB inputs, circumventing the limitations of traditional 2D-to-3D lifting methods. By leveraging a feed-forward metric 3D reconstruction model as its geometric backbone, Map-Det3D effectively integrates detection into a reconstructed 3D space, enhancing robustness against domain shifts. Experimental results demonstrate that this approach achieves strong online performance and stable metric 3D detection without the need for adaptation across various benchmarks.
Monocular 3D object detection can achieve robust performance without the pitfalls of 2D-to-3D lifting, thanks to a novel integration of metric reconstruction and detection.
Metric 3D object detection is a core capability for embodied agents, yet most reliable systems lean on depth sensors, trading away cost, power, and integration simplicity. This motivates monocular 3D detection, which avoids additional constraints, yet it faces a major obstacle: from a single image, depth, and especially absolute scale, are underconstrained. As a result, the prevailing pattern of detecting in 2D and then predicting 3D attributes is often brittle, since modest range errors can dominate 3D localization, and the learned scale prior can fail when cameras, motion, or environments undergo domain shifts. To address this, we propose Map-Det3D, an online multi-view 3D object detection model that brings detection directly into a 3D space reconstructed from RGB. We map a short temporal window into multiple views and repurpose a feed-forward metric 3D reconstruction model as our geometric backbone while tuning its object-aware capabilities. Building on this representation, Map-Det3D directly predicts boxes in metric 3D space, without the widely used 2D-to-3D lifting. Experiments across different benchmarks show that this design supports strong online performance and robust transfer without adaptation, suggesting that training reconstruction priors for detection is a practical route to stable metric 3D detection from monocular video. Code and models are available at https://royyang0714.github.io/Map-Det3D.
