Search papers, labs, and topics across Lattice.
This paper explores efficient sampling techniques for empirical pairwise loss functions, which are critical in machine learning tasks like similarity learning and ranking but suffer from high computational costs. By employing survey sampling methods that focus on informative pairs rather than individual observations, the authors demonstrate that significant reductions in computational load can be achieved without sacrificing performance. Their findings reveal that targeted sampling can yield results comparable to full pairwise evaluations, offering a valuable trade-off between accuracy and efficiency in high-dimensional settings such as vision and graph learning.
Targeted sampling of informative pairs can slash computational costs in pairwise loss estimation while maintaining performance on par with full evaluations.
Many machine learning problems, including similarity learning, ranking, and clustering, rely on empirical pairwise loss functions whose quadratic computational cost quickly becomes prohibitive at scale. We demonstrate how a frugal approach that retains only a fraction of the available information on pairs can achieve estimation or optimization performance comparable to that obtained by using all pairs, by leveraging survey sampling techniques. A central finding, supported by both theory and experiments, is that such sampling plans must target pairs directly rather than individual observations. In particular, for pairwise losses between high-dimensional vectors such as embeddings in vision or graph learning, assigning higher inclusion probabilities to informative pairs using suitable auxiliary information yields performance close to full pairwise evaluation, providing a principled and theoretically grounded trade-off between accuracy and computational cost.
