Search papers, labs, and topics across Lattice.
This paper introduces Stepwise Semantic Alignment (SSA), a novel test-time adaptation method that uses a pseudo-source domain as a semantic bridge between the unknown source and unlabeled target domains. SSA rectifies the semantic features of the pseudo-source using universal semantics and then aligns the target domain with the corrected pseudo-source. The method also incorporates a Hierarchical Feature Aggregation module and a Confidence-Aware Complementary Learning strategy to improve semantic quality, achieving a 5.2% performance boost on GTA2Cityscapes for semantic segmentation.
Instead of directly aligning to a flawed pseudo-source domain in test-time adaptation, a semantic bridge approach significantly boosts performance by first rectifying the pseudo-source using universal semantics.
Distribution shifts between training and testing data are a critical bottleneck limiting the practical utility of models, especially in real-world test-time scenarios. To adapt models when the source domain is unknown and the target domain is unlabeled, previous works constructed pseudo-source domains via data generation and translation, then aligned the target domain with them. However, significant discrepancies exist between the pseudo-source and the original source domain, leading to potential divergence when correcting the target directly. From this perspective, we propose a Stepwise Semantic Alignment (SSA) method, viewing the pseudo-source as a semantic bridge connecting the source and target, rather than a direct substitute for the source. Specifically, we leverage easily accessible universal semantics to rectify the semantic features of the pseudo-source, and then align the target domain using the corrected pseudo-source semantics. Additionally, we introduce a Hierarchical Feature Aggregation (HFA) module and a Confidence-Aware Complementary Learning (CACL) strategy to enhance the semantic quality of the SSA process in the absence of source and ground truth of target domains. We evaluated our approach on tasks like semantic segmentation and image classification, achieving a 5.2% performance boost on GTA2Cityscapes over the state-of-the-art.
