Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper tackles the problem of data distribution valuation, quantifying the value of data distributions based on samples, using a novel Generalized Bayes Valuation framework. The framework leverages generalized Bayesian inference with a loss function based on transferability measures, unifying approaches to annotator evaluation and data augmentation. Experiments demonstrate the framework's effectiveness and efficiency in real-world scenarios, including extensions to continuous data streams.
Discover how a Bayesian approach can unify annotator evaluation and data augmentation by valuing data distributions based on transferability.
We investigate the data distribution valuation problem, which aims to quantify the values of data distributions from their samples. This is a recently proposed problem that is related to but different from classical data valuation and can be applied to various applications. For this problem, we develop a novel framework called Generalized Bayes Valuation that utilizes generalized Bayesian inference with a loss constructed from transferability measures. This framework allows us to solve, in a unified way, seemingly unrelated practical problems, such as annotator evaluation and data augmentation. Using the Bayesian principles, we further improve and enhance the applicability of our framework by extending it to the continuous data stream setting. Our experiment results confirm the effectiveness and efficiency of our framework in different real-world scenarios.