Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Paired Sampling for Domain Adaptation (PSDA), a novel stochastic variance reduction technique aimed at addressing the high variance issues in unsupervised domain adaptation (UDA) losses like correlation alignment and maximum mean discrepancy. By pairing observations within and across domains to create quadruplets, PSDA minimizes expected gradient variance, which enhances the effectiveness of minibatch optimization. Experimental results on three domain shift datasets reveal that PSDA significantly improves target domain accuracy compared to existing methods.
Pairing observations in domain adaptation can drastically reduce gradient variance and boost accuracy, outperforming traditional methods.
Correlation alignment and the maximum mean discrepancy are two widely used distribution-matching frameworks for unsupervised domain adaptation (UDA). However, high variance in these losses has been shown to undermine their effectiveness in minibatch optimisation settings. Furthermore, the losses lack finite-sum structure, which renders them incompatible with classical stochastic variance reduction (SVR) methods. This paper proposes Paired Sampling for Domain Adaptation (PSDA), a novel SVR technique tailored to such objectives. PSDA pairs observations both within and across domains, to form quadruplets that are always sampled together during training. The pairings are designed to minimise expected gradient variance, and reduce to solving a set of linear assignment problems. Our simulations demonstrate reduced variance compared to related methods, and experiments on three domain shift datasets show improved target domain accuracy.