Search papers, labs, and topics across Lattice.
This paper investigates the collateral knowledge damage caused by unlearning in large language models (LLMs), revealing that the impact of forgetting specific information can propagate to both same-domain and distant-domain knowledge. The authors identify a consistent decay pattern in collateral damage, which is strongest near the forget set and diminishes with semantic distance, but still affects knowledge across domain boundaries. They also introduce a pre-unlearning auditing method that leverages interaction features to predict potential damage, providing a framework for safer unlearning practices in LLMs.
Unlearning in LLMs can inadvertently damage up to 40% of related knowledge, but a new auditing method can predict this collateral damage before it happens.
Machine unlearning for large language models (LLMs) aims to remove specified knowledge while preserving the rest of the model's capabilities. However, the boundary between knowledge to forget and knowledge to retain is often unclear, since related and even distant information may be entangled in the model. In this paper, we study LLM unlearning from a data-centric perspective and measure how unlearning effects propagate from the forget set to same-domain and distant-domain knowledge. We find a consistent decay pattern: collateral damage is strongest near the forget set, weakens with semantic distance, but does not disappear at domain boundaries. We further ask whether such damage can be audited before unlearning is executed. We formulate forget-set auditing as a pre-unlearning prediction task and analyze which data features are most predictive of downstream damage. Our results show that interaction features between the forget set and evaluation set provide the strongest signals, suggesting that collateral damage is partly reflected in data geometry before model updates occur. These findings position forget-set auditing as an early warning tool for identifying risky unlearning runs and designing more reliable unlearning procedures.
