Search papers, labs, and topics across Lattice.
This paper introduces a novel unlearning algorithm, Hardness-Aware Multi-Objective Unlearning (HAMU), which quantifies the difficulty of reconciling the objectives of forgetting specific data while retaining model performance. By employing a constrained optimization approach, HAMU guarantees a specified improvement in forget quality while minimizing degradation in retain utility, informed by a hardness measure that assesses the similarity between forget and retain data. Empirical results show that HAMU outperforms existing unlearning methods on both image and text datasets, demonstrating its effectiveness in real-world applications.
Forgetting specific data without sacrificing model performance just got a theoretical boost鈥擧AMU guarantees improvements in unlearning quality based on data hardness.
Machine unlearning aims to remove the influence of specific forget training data due to privacy, copyright or bias concerns while maintaining the model performance on the remaining retain data. Existing unlearning algorithms, such as optimizing a weighted combination of losses, have tried to achieve these objectives of improving forget quality and maintaining retain utility. However, they do not guarantee that these objectives can be improved by a specified extent for all forget and retain data. In this work, we address this limitation with a novel and theoretically-grounded approach from a constrained optimization perspective. Firstly, we identify that the hardness of reconciling both objectives can be quantified by the similarity between the forget data and the retain data. Next, we derive an unlearning algorithm (HAMU) with the overall goal of guaranteeing a specified improvement in forget quality while minimizing the retain utility cost/degradation by updating the model weights based on our hardness measure. Our hardness measure also informs users when retain utility degradation is unavoidable, i.e., both objectives cannot be improved simultaneously, and stopping should be considered. Our algorithm is applicable to non-convex models and is easily parallelizable, making it readily deployable in real-world scenarios. We empirically demonstrate HAMU's superior performance over baselines on both image and text datasets using large models. Our code is available at https://github.com/aoi3142/HAMU.
