Search papers, labs, and topics across Lattice.
This paper introduces Easy Data Unlearning Bench, a benchmarking suite designed to simplify the evaluation of machine unlearning algorithms. The benchmark uses the KLoM (KL divergence of Margins) metric and provides precomputed model ensembles and oracle outputs to reduce setup complexity. The authors demonstrate the framework's utility in enabling reproducible and scalable comparisons across various unlearning methods.
Stop wrestling with complex setups: this new benchmark makes evaluating machine unlearning algorithms as easy as running a script.
Evaluating machine unlearning methods remains technically challenging, with recent benchmarks requiring complex setups and significant engineering overhead. We introduce a unified and extensible benchmarking suite that simplifies the evaluation of unlearning algorithms using the KLoM (KL divergence of Margins) metric. Our framework provides precomputed model ensembles, oracle outputs, and streamlined infrastructure for running evaluations out of the box. By standardizing setup and metrics, it enables reproducible, scalable, and fair comparison across unlearning methods. We aim for this benchmark to serve as a practical foundation for accelerating research and promoting best practices in machine unlearning. Our code and data are publicly available.
DeepMind