Search papers, labs, and topics across Lattice.
This paper introduces HilDA, a self-supervised pretraining framework that enhances LiDAR backbones by leveraging hierarchical distillation and temporal occupancy diffusion to better capture semantic and geometric information for autonomous driving tasks. By addressing the limitations of existing methods that treat Vision Foundation Models as black boxes, HilDA employs multi-layer distillation and global context distillation to improve the alignment of features across different layers. The results demonstrate that models pre-trained with HilDA achieve state-of-the-art performance on cross-modal distillation benchmarks, significantly outperforming previous approaches in 3D object detection, scene flow, and semantic occupancy prediction.
Hierarchical distillation with temporal occupancy diffusion enables LiDAR models to achieve unprecedented accuracy in 3D perception tasks.
Leveraging Vision Foundation Models (VFMs) for camera-to-LiDAR knowledge distillation offers a promising solution to the scarcity of annotated data needed to represent the immense geometric and kinematic diversity of real-world autonomous driving (AD). However, current approaches typically treat VFMs as black-box teachers, relying exclusively on frame-wise feature similarity. Consequently, they do not fully exploit the teacher's layer-wise semantic structure and global context, as well as the rich spatiotemporal information inherent in LiDAR sequences. We propose HilDA, a self-supervised pretraining framework for LiDAR backbones that better captures the semantic what and geometric where needed for driving tasks. HilDA combines hierarchical distillation comprising multi-layer distillation for progressive semantic alignment and global context distillation for scene-level semantics, with a temporal occupancy diffusion objective promoting spatiotemporal consistency. Models pre-trained with HilDA achieve state-of-the-art results on cross-modal distillation benchmarks and outperform models trained via prior distillation approaches on 3D object detection, scene flow, and semantic occupancy prediction. Code available at: https://maxiuw.github.io/hilda.
