Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates machine unlearning by applying the concept of mode connectivity, which examines how independently trained models can be linked by smooth paths in the parameter space. The authors introduce a novel framework called mode connectivity in unlearning (MCU), revealing that unlearned models often reside in connected basins with varying privacy metrics and nonlinear unlearning progressions. Key findings indicate that the training dynamics can shift models between basins and that MCU-based ensembling enhances generalization and resilience against relearning attacks.
Unlearning isn't just about erasing data—it's a complex journey through parameter space that can significantly impact model privacy and performance.
Machine Unlearning aims to remove undesired information from trained models without full retraining from scratch. Despite recent progress, the loss landscape and optimization geometry of unlearning are poorly understood. In this paper, we study machine unlearning through the lens of mode connectivity--the phenomenon that independently trained models can often be connected by smooth low-loss paths in parameter space. We introduce {\em mode connectivity in unlearning} (MCU) and evaluate it across a range of settings, including curriculum learning, second-order optimization, and connectivity across different unlearning methods. We find that many unlearned models lie in connected basins with smooth retain/forget behavior, while changes in training dynamics can move solutions into different basins. MCU also reveals that models within the same basin can differ substantially on privacy metrics, and that unlearning progresses nonlinearly from the original model to the unlearned model. In addition, linear connectivity suggests that most approximate unlearning methods are mechanistically distinct from retraining. Finally, MCU-based ensembling can improve generalization and robustness to relearning attacks, and MCU smoothness correlates with unlearning difficulty. To our knowledge, this is the first study of machine unlearning through the lens of mode connectivity.