Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Noise-Conditional Gated Rectification (NCGR) to address the degradation in bird's-eye-view (BEV) 3D object detection caused by extrinsic perturbations in camera projections. By predicting a 2D rectification offset for each query-camera pair and employing a camera-level gating mechanism, NCGR effectively compensates for projection errors without requiring full extrinsic corrections. Evaluated on the nuScenes dataset, NCGR outperforms existing methods like BEVFormer and CAPE under dynamic perturbations, achieving a significant improvement in detection performance with a 39.69% NDS score.
NCGR compensates for camera projection errors in BEV 3D detection, achieving a remarkable 39.69% NDS in challenging dynamic conditions.
Camera-based bird's-eye-view (BEV) 3D detection typically assumes accurate and fixed camera extrinsics. In detectors using spatial cross-attention (SCA), extrinsic perturbations displace the image-plane projections of BEV reference points, causing queries to sample features from incorrect regions and degrading detection performance. To address this failure mode, Noise-Conditional Gated Rectification (NCGR) is proposed to compensate for projection errors without explicitly estimating a full six-degree-of-freedom extrinsic correction. For each query-camera pair, a 2D rectification offset is predicted and modulated by a camera-level gate to rectify the base projection before native deformable sampling. During training, the perturbation-derived quantities used to construct the condition and gate are gradually replaced through scheduled interpolation by counterparts generated from an auxiliary scalar predicted from camera features. This transition enables blind inference without perturbation metadata. During training, a weight-shared clean-teacher/perturbed-student pair is used, and the rectification module is supervised by a BEV-consistency objective between the two branches. NCGR is evaluated on nuScenes with simulated dynamic and static extrinsic perturbations. In a five-camera dynamic stress test, NCGR achieves 39.69% NDS, compared with 28.00% for BEVFormer and 33.23% for CAPE. Under clean extrinsics, NCGR maintains performance comparable to that of BEVFormer.