Search papers, labs, and topics across Lattice.
The paper introduces AW-MoE, a Mixture of Experts framework for robust multi-modal 3D object detection under adverse weather conditions, which addresses performance conflicts arising from data distribution discrepancies across different weather scenarios. AW-MoE uses Image-guided Weather-aware Routing (IWR) to classify weather conditions based on image features and selects the top-K most relevant Weather-Specific Experts (WSE). The framework also incorporates a Unified Dual-Modal Augmentation (UDMA) for LiDAR and 4D Radar data, achieving a 15% improvement in adverse weather performance over existing methods with minimal overhead.
Autonomous vehicles can now see through the storm: a new Mixture of Experts approach boosts 3D object detection accuracy by 15% in adverse weather, without slowing things down.
Robust 3D object detection under adverse weather conditions is crucial for autonomous driving. However, most existing methods simply combine all weather samples for training while overlooking data distribution discrepancies across different weather scenarios, leading to performance conflicts. To address this issue, we introduce AW-MoE, the framework that innovatively integrates Mixture of Experts (MoE) into weather-robust multi-modal 3D object detection approaches. AW-MoE incorporates Image-guided Weather-aware Routing (IWR), which leverages the superior discriminability of image features across weather conditions and their invariance to scene variations for precise weather classification. Based on this accurate classification, IWR selects the top-K most relevant Weather-Specific Experts (WSE) that handle data discrepancies, ensuring optimal detection under all weather conditions. Additionally, we propose a Unified Dual-Modal Augmentation (UDMA) for synchronous LiDAR and 4D Radar dual-modal data augmentation while preserving the realism of scenes. Extensive experiments on the real-world dataset demonstrate that AW-MoE achieves ~ 15% improvement in adverse-weather performance over state-of-the-art methods, while incurring negligible inference overhead. Moreover, integrating AW-MoE into established baseline detectors yields performance improvements surpassing current state-of-the-art methods. These results show the effectiveness and strong scalability of our AW-MoE. We will release the code publicly at https://github.com/windlinsherlock/AW-MoE.
