Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ARROW, an innovative online stochastic variance reduction algorithm tailored for the maximum mean discrepancy and correlation alignment loss functions, addressing the limitations of existing offline methods in dynamic learning environments. By maintaining moving average references of alignment statistics and adaptively reweighting incoming minibatches, ARROW ensures that the statistics remain aligned, facilitating effective learning from streaming data. Experimental results demonstrate that ARROW achieves competitive performance compared to offline algorithms in terms of runtime efficiency, variance reduction, and accuracy in target domains.
ARROW achieves online variance reduction for domain adaptation, outperforming traditional offline methods in dynamic data environments.
This paper studies the problem of stochastic variance reduction (SVR) for the maximum mean discrepancy (MMD) and correlation alignment (CORAL) loss functions. Although various offline SVR algorithms for these losses have been proposed, these are incompatible with online, distributed, or incremental learning settings. This paper presents Adaptive vaRiance Reduction via Online reWeighting (ARROW), the first online SVR algorithm for the MMD and CORAL for streamed data. The method maintains moving average references of the alignment statistics, and adaptively reweights incoming minibatches so that the minibatch and reference statistics are aligned. Further, we propose a relaxed reweighting scheme so that the ensuing weight-optimisation problem is tractable. In experiments and simulations, we show that ARROW performs competitively with offline algorithms in terms of runtime, degree of variance reduction achieved, and target domain accuracy.