Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Distortion-Aware PETR (DAPETR), a novel projection-free detector specifically designed for mixed pinhole-fisheye camera setups in 3D object detection. By integrating a unified distortion-aware positional embedding and a bidirectional feature-geometry co-modulation module, DAPETR effectively addresses the challenges posed by severe radial distortion, which typically undermines the performance of conventional BEV detectors. Experimental results on the KITTI-360 benchmark demonstrate that DAPETR outperforms existing methods, revealing critical interactions between learned adaptations and geometric reparameterization that can hinder performance when combined.
DAPETR reveals that learned adaptations can conflict with geometric reparameterization, significantly enhancing 3D object detection from fisheye cameras.
Fisheye cameras are widely deployed in autonomous driving perception suites for their low cost and full-coverage field of view (FOV), yet their potential remains underleveraged in 3D object detection. Severe radial distortion challenges most BEV detectors by violating the fundamental assumption of uniform sampling. To bridge this gap, we propose Distortion-Aware PETR (DAPETR), a projection-free detector tailored for mixed pinhole-fisheye camera setups. DAPETR incorporates two key learned-adaptive modules: a unified distortion-aware positional embedding that harmonizes positional encodings for image representations with fisheye geometry, and a bidirectional feature-geometry co-modulation module that mutually adapts image features and 3D positional embeddings. In our experiments on a converted KITTI-360 benchmark, we systematically compare our learned adaptive approach against PETR in polar coordinates (PolarPETR). We find that while both methods improve over the baseline, our learned modules achieve superior performance. Crucially, we uncover a negative interaction when combining both strategies, revealing that learned adaptation and explicit geometric reparameterization can conflict. Our final DAPETR model significantly advances the research and benchmark for fisheye BEV detection, providing critical insights into effective distortion-aware 3D perception design other than image rectification.