Search papers, labs, and topics across Lattice.
The paper addresses the challenge of generalizing multi-camera 3D object detection (MC3D) models to new platforms with unseen camera configurations. They identify spatial prior discrepancies (intrinsics, extrinsics, array layouts) as the key issue and propose CoIn3D, a framework incorporating spatial-aware feature modulation (SFM) and camera-aware data augmentation (CDA). Experiments on NuScenes, Waymo, and Lyft datasets demonstrate CoIn3D's strong cross-configuration performance across different MC3D paradigms.
MC3D models can now generalize to unseen camera configurations thanks to a new framework that explicitly accounts for spatial prior discrepancies.
Multi-camera 3D object detection (MC3D) has attracted increasing attention with the growing deployment of multi-sensor physical agents, such as robots and autonomous vehicles. However, MC3D models still struggle to generalize to unseen platforms with new multi-camera configurations. Current solutions simply employ a meta-camera for unified representation but lack comprehensive consideration. In this paper, we revisit this issue and identify that the devil lies in spatial prior discrepancies across source and target configurations, including different intrinsics, extrinsics, and array layouts. To address this, we propose CoIn3D, a generalizable MC3D framework that enables strong transferability from source configurations to unseen target ones. CoIn3D explicitly incorporates all identified spatial priors into both feature embedding and image observation through spatial-aware feature modulation (SFM) and camera-aware data augmentation (CDA), respectively. SFM enriches feature space by integrating four spatial representations, such as focal length, ground depth, ground gradient, and Pl\"ucker coordinate. CDA improves observation diversity under various configurations via a training-free dynamic novel-view image synthesis scheme. Extensive experiments demonstrate that CoIn3D achieves strong cross-configuration performance on landmark datasets such as NuScenes, Waymo, and Lyft, under three dominant MC3D paradigms represented by BEVDepth, BEVFormer, and PETR.
Amazon Science