Search papers, labs, and topics across Lattice.
The paper introduces LiREC-Net, a novel target-free, learning-based calibration network designed for the joint calibration of LiDAR, RGB, and event data. It addresses the limitation of existing learning-based methods that typically handle only bi-modal sensor setups by proposing a unified framework for tri-modal calibration. The network employs a shared LiDAR representation, leveraging both 3D and projected depth map features, to enhance consistency and reduce computational redundancy.
Jointly calibrating LiDAR, RGB, and event data is now possible with LiREC-Net, a single network that rivals the accuracy of specialized bi-modal methods.
Advanced autonomous systems rely on multi-sensor fusion for safer and more robust perception. To enable effective fusion, calibrating directly from natural driving scenes (i.e., target-free) with high accuracy is crucial for precise multi-sensor alignment. Existing learning-based calibration methods are typically designed for only a single pair of sensor modalities (i.e., a bi-modal setup). Unlike these methods, we propose LiREC-Net, a target-free, learning-based calibration network that jointly calibrates multiple sensor modality pairs, including LiDAR, RGB, and event data, within a unified framework. To reduce redundant computation and improve efficiency, we introduce a shared LiDAR representation that leverages features from both its 3D nature and projected depth map, ensuring better consistency across modalities. Trained and evaluated on established datasets, such as KITTI and DSEC, our LiREC-Net achieves competitive performance to bi-modal models and sets a new strong baseline for the tri-modal use case.
