Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces ClimaDrive, a semantics-guided image-to-image translation framework, to generate physically realistic and weather-diverse synthetic data for training anomaly segmentation models. ClimaDrive combines structure-guided multi-weather generation with prompt-driven anomaly inpainting to create visually realistic training data. Experiments on the newly created ClimaOoD benchmark demonstrate that training with this synthetic data significantly improves the performance of state-of-the-art anomaly segmentation methods, as evidenced by improvements in AUROC, AP, and FPR95 metrics.
Synthesizing physically plausible anomalies with weather variations closes the sim-to-real gap, boosting anomaly segmentation performance where other synthetic data methods fall short.
Anomaly segmentation seeks to detect and localize unknown or out-of-distribution (OoD) objects that fall outside predefined semantic classes a capability essential for safe autonomous driving. However, the scarcity and limited diversity of anomaly data severely constrain model generalization in open-world environments. Existing approaches mitigate this issue through synthetic data generation, either by copy-pasting external objects into driving scenes or by leveraging text-to-image diffusion models to inpaint anomalous regions. While these methods improve anomaly diversity, they often lack contextual coherence and physical realism, resulting in domain gaps between synthetic and real data. In this paper, we present ClimaDrive, a semantics-guided image-to-image framework for synthesizing semantically coherent, weather-diverse, and physically plausible OoD driving data. ClimaDrive unifies structure-guided multi-weather generation with prompt-driven anomaly inpainting, enabling the creation of visually realistic training data. Based on this framework, we construct ClimaOoD, a large-scale benchmark spanning six representative driving scenarios under both clear and adverse weather conditions. Extensive experiments on four state-of-the-art methods show that training with ClimaOoD leads to robust improvements in anomaly segmentation. Across all methods, AUROC, AP, and FPR95 show notable gains, with FPR95 dropping from 3.97 to 3.52 for RbA on Fishyscapes LAF. These results demonstrate that ClimaOoD enhances model robustness, offering valuable training data for better generalization in open-world anomaly detection.