Search papers, labs, and topics across Lattice.
This paper introduces Ultra, an Unsupervised Restoration-Segmentation Collaborative Learning Framework designed to enhance semantic segmentation in adverse weather conditions by effectively transferring knowledge from normal-weather images. The framework addresses the challenge of hallucination-driven error propagation by reframing cross-task optimization as direction selection under uncertainty and causal effect estimation, employing two novel components: CTDN for cooperative direction selection and CMIL for causal effect assessment. Extensive experiments across three UDA-ASS benchmarks demonstrate that Ultra achieves state-of-the-art segmentation performance while also improving unsupervised restoration results and generalizing to other collaboration tasks like object detection.
Hallucination-driven error propagation is tackled head-on, enabling state-of-the-art segmentation performance in adverse weather conditions through innovative cross-task collaboration.
Unsupervised Domain Adaptation for Adverse Weather Semantic Segmentation (UDA-ASS) aims to transfer semantic knowledge from labeled normal-weather images to unlabeled adverse environments. Existing approaches implicitly assume that restoration and segmentation provide mutually beneficial guidance. However, under severe degradation and without target-domain supervision, the validity of cross-task optimization directions becomes fundamentally unidentifiable, leading to hallucination-driven error propagation. In this work, we propose a novel Unsupervised Restoration-Segmentation Collaborative Learning Framework (Ultra), which reframes cross-task interaction as direction selection under uncertainty and causal effect estimation, enabling reliable collaboration through candidate direction generation and intervention-based filtering. In detail, we propose CTDN and CMIL. The former exploits complementary visual structures and semantic information to generate candidate optimization directions and performs cooperative direction selection between restoration and segmentation. The latter reformulates cross-task information transfer from correlation-based propagation into causal effect assessment, suppressing hallucination propagation. Extensive experiments on three widely used UDA-ASS benchmarks demonstrate state-of-the-art segmentation performance. Beyond segmentation, our framework achieves better unsupervised restoration results than existing UDA-ASS restoration methods and generalizes to unsupervised restoration and object detection collaboration tasks. Code and models will be available at https://github.com/Wang-Shiqin/Ultra.
