Search papers, labs, and topics across Lattice.
Fusion-Poly is introduced as a spatial-temporal fusion framework for 3D multi-object tracking (MOT) that leverages both synchronized and asynchronous LiDAR and camera data. It addresses the limitation of existing methods that underutilize asynchronous observations due to different sensor sampling rates. The framework incorporates frequency-aware cascade matching, trajectory estimation, and full-state observation alignment modules to achieve higher-frequency updates and improved cross-modal consistency. Fusion-Poly achieves a new state-of-the-art AMOTA of 76.5% on the nuScenes test set.
Unlock more robust 3D multi-object tracking by fusing asynchronous LiDAR and camera data, achieving state-of-the-art results on nuScenes.
LiDAR-camera 3D multi-object tracking (MOT) combines rich visual semantics with accurate depth cues to improve trajectory consistency and tracking reliability. In practice, however, LiDAR and cameras operate at different sampling rates. To maintain temporal alignment, existing data pipelines usually synchronize heterogeneous sensor streams and annotate them at a reduced shared frequency, forcing most prior methods to perform spatial fusion only at synchronized timestamps through projection-based or learnable cross-sensor association. As a result, abundant asynchronous observations remain underexploited, despite their potential to support more frequent association and more robust trajectory estimation over short temporal intervals. To address this limitation, we propose Fusion-Poly, a spatial-temporal fusion framework for 3D MOT that integrates asynchronous LiDAR and camera data. Fusion-Poly associates trajectories with multi-modal observations at synchronized timestamps and with single-modal observations at asynchronous timestamps, enabling higher-frequency updates of motion and existence states. The framework contains three key components: a frequency-aware cascade matching module that adapts to synchronized and asynchronous frames according to available detection modalities; a frequency-aware trajectory estimation module that maintains trajectories through high-frequency motion prediction, differential updates, and confidence-calibrated lifecycle management; and a full-state observation alignment module that improves cross-modal consistency at synchronized timestamps by optimizing image-projection errors. On the nuScenes test set, Fusion-Poly achieves 76.5% AMOTA, establishing a new state of the art among tracking-by-detection 3D MOT methods. Extensive ablation studies further validate the effectiveness of each component. Code will be released.
