Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a Geometry-Aware Hybrid Fusion (GA-HF) framework that optimally integrates dual-fisheye camera and LiDAR data for 3D object detection in low-overlap scenarios. By employing a Distortion-Aware Lift-Splat-Shoot (LSS) module to convert fisheye features into a polar BEV grid and a Dual-Attention Warping Correction module to enhance feature alignment, the method effectively mitigates the geometric challenges posed by fisheye optics. The results demonstrate a significant performance improvement, with GA-HF achieving a 4.2% increase in NDS on the KITTI-360 benchmark and outperforming existing methods on other datasets, marking a notable advancement in sensor fusion techniques.
GA-HF not only boosts 3D object detection accuracy by over 4% but also tackles the unique challenges of fisheye-LiDAR integration, setting a new standard for sensor fusion in autonomous systems.
As autonomous systems expand from capital-intensive robotaxis to cost-sensitive logistics, sensor configurations are increasingly optimized for coverage-per-cost. A prevalent sparse-view setup utilizes dual-fisheye cameras with a roof-mounted LiDAR, introducing severe geometric challenges: extreme radial distortion, minimal overlap, and misalignment between spherical projections and rectilinear grids. BEV fusion algorithms typically force image and point cloud modalities into unified Cartesian grids early in the pipeline, causing significant feature distortion and information loss for wide-view fisheye cameras. To address this, we propose a Geometry-Aware Hybrid Fusion (GA-HF) framework that explicitly accounts for fisheye geometry and BEV feature distortion, where fisheye features are lifted into a polar BEV grid via a Distortion-Aware Lift-Splat-Shoot (LSS) module to preserve native angular density, while LiDAR features are processed in native Cartesian space for metric fidelity of bounding box regression. To bridge these heterogeneous streams, we introduce a Dual-Attention Warping Correction module that applies spatial and channel attention to the warped camera features before fusion, explicitly suppressing artifacts in low-quality peripheral regions while enhancing high-quality semantic cues. GA-HF is evaluated on three benchmarks: KITTI-360, Dur360BEV, and Fisheye3DOD datasets. To the best of our knowledge, it is the first approach to explore LiDAR-fisheye camera fusion. On KITTI-360, GA-HF improves NDS by 4.2% over Cartesian baselines; on Dur360BEV, it surpasses both LiDAR-only and BEVFusion, while significantly reducing orientation error despite the geometric distortions; on Fisheye3DOD, it attains the highest detection score among all fusion methods.