Search papers, labs, and topics across Lattice.
This paper introduces a cross-modal pseudo-labeling pipeline that facilitates unsupervised domain adaptation for semantic segmentation without the need for target-domain annotations. By leveraging the SAM model for class-agnostic region proposals and EVA-CLIP for semantic label assignment based on region-text similarity, the approach ensures that only high-confidence pseudo-labels are used for self-training. The findings reveal that the quality of pseudo-labels significantly impacts performance in domain shifts, particularly in applications like industrial waste sorting and autonomous driving, outperforming traditional source-only methods.
Pseudo-label quality, not quantity, is the key to successful self-training in domain adaptation, revealing a critical insight for real-world applications like waste sorting.
Obtaining labeled data for semantic segmentation in applied settings (e.g., autonomous driving, industrial waste sorting) is expensive and often infeasible at scale. We present a cross-modal pseudo-labeling pipeline that enables unsupervised domain adaptation without any target-domain annotations. The pipeline is built on two core foundation models: SAM generates class-agnostic region proposals, and EVA-CLIP assigns semantic labels based on region-text similarity, with confidence filtering ensuring that only reliable pseudo-labels are used for self-training a segmentation model. As an optional extension, BLIP provides language-grounded verification for ambiguous regions, thereby improving pseudo-label quality without altering the overall pipeline. Evaluated on two domain shifts, synthetic-to-real autonomous driving and, with a primary focus, lab-to-factory industrial waste sorting, the pipeline consistently improves over source-only baselines. Our results demonstrate that pseudo-label quality, not quantity, is a decisive factor in self-training under domain shift, and that cross-modal language grounding offers a practical path to reliable automatic annotation in deployment-critical applications.
