Search papers, labs, and topics across Lattice.
This paper introduces PanDA, a novel unsupervised domain adaptation (UDA) framework for multimodal 3D panoptic segmentation in autonomous driving, addressing the challenge of domain shifts in real-world scenarios. PanDA employs asymmetric multimodal augmentation to improve robustness against single-sensor degradation and a dual-expert pseudo-label refinement module to enhance pseudo-label completeness and reliability. Experiments across diverse domain shifts demonstrate that PanDA significantly outperforms state-of-the-art UDA baselines for 3D semantic segmentation, improving generalization in autonomous driving perception.
Autonomous vehicles can now better handle degraded sensor data in adverse conditions thanks to a new unsupervised domain adaptation method that leverages both LiDAR and RGB data.
This paper presents the first study on Unsupervised Domain Adaptation (UDA) for multimodal 3D panoptic segmentation (mm-3DPS), aiming to improve generalization under domain shifts commonly encountered in real-world autonomous driving. A straightforward solution is to employ a pseudo-labeling strategy, which is widely used in UDA to generate supervision for unlabeled target data, combined with an mm-3DPS backbone. However, existing supervised mm-3DPS methods rely heavily on strong cross-modal complementarity between LiDAR and RGB inputs, making them fragile under domain shifts where one modality degrades (e.g., poor lighting or adverse weather). Moreover, conventional pseudo-labeling typically retains only high-confidence regions, leading to fragmented masks and incomplete object supervision, which are issues particularly detrimental to panoptic segmentation. To address these challenges, we propose PanDA, the first UDA framework specifically designed for multimodal 3D panoptic segmentation. To improve robustness against single-sensor degradation, we introduce an asymmetric multimodal augmentation that selectively drops regions to simulate domain shifts and improve robust representation learning. To enhance pseudo-label completeness and reliability, we further develop a dual-expert pseudo-label refinement module that extracts domain-invariant priors from both 2D and 3D modalities. Extensive experiments across diverse domain shifts, spanning time, weather, location, and sensor variations, significantly surpass state-of-the-art UDA baselines for 3D semantic segmentation.
