Search papers, labs, and topics across Lattice.
This paper introduces a variance-based measure to quantify heterogeneity in supervised learning training data, capturing the influence of data point pairs. They demonstrate that this variance measure reflects data heterogeneity, enabling the identification of mixed distributions within a dataset. By partitioning data into blocks based on this variance measure and training models on purified blocks, they achieve significant improvements in test accuracy, validated on EMNIST image data and synthetic datasets.
Data heterogeneity, quantified by a novel variance measure, can be exploited to partition training data and substantially boost supervised learning accuracy.
In this article the authors develop an intrinsic measure for quantifying heterogeneity in training data for supervised learning. This measure is the variance of a random variable which factors through the influences of pairs of training points. The variance is shown to capture data heterogeneity and can thus be used to assess if a sample is a mixture of distributions. The authors prove that the data itself contains key information that supports a partitioning into blocks. Several proof of concept studies are provided that quantify the connection between variance and heterogeneity for EMNIST image data and synthetic data. The authors establish that variance is maximal for equal mixes of distributions, and detail how variance-based data purification followed by conventional training over blocks can lead to significant increases in test accuracy.
