Search papers, labs, and topics across Lattice.
This paper introduces GROM, a novel one-shot machine unlearning method that eliminates sensitive knowledge from large language models without relying on iterative optimization. By framing the unlearning process as a ridge-regularized least-squares optimization problem, GROM computes a closed-form additive update for weight matrices that effectively suppresses unwanted content while preserving performance on retained data. Extensive evaluations show that GROM significantly outperforms traditional fine-tuning methods in forgetting-utility trade-offs, achieving rapid unlearning in seconds and resisting quantization attacks that can recover forgotten content.
GROM achieves rapid, effective unlearning in seconds while maintaining model performance, outpacing traditional methods that struggle with computational efficiency and content recovery.
Machine unlearning has become a critical capability for safely removing specific, sensitive knowledge from large language models (LLMs). Current state-of-the-art approaches primarily rely on iterative, training-time unlearning via fine-tuning. However, even when utilizing parameter-efficient dimensionality reduction techniques like LoRA, gradient-based optimization remains computationally expensive and lacks explicit analytical formulations. It can also leave the targeted knowledge merely hidden rather than removed, to the point that simply quantizing the unlearned model restores much of what it was supposed to have erased. To resolve this, we propose a novel one-shot unlearning approach, abandoning iterative optimization in favor of a direct, exact analytical solution. We frame the unlearning process as a ridge-regularized least-squares optimization problem, deriving a closed-form additive update for targeted weight matrices. This update forces the selected layer to suppress unwanted content while strictly preserving its behavior on retained data. Computed from gradient-free forward passes alone, with no backpropagation and no iteration to convergence, GROM applies the weight edit in mere seconds, which makes it orders of magnitude faster than traditional fine-tuning. Extensive evaluations demonstrate that GROM achieves state-of-the-art forgetting-utility trade-offs on TOFU-5%, TOFU-10%, MUSE-Books, MUSE-News and WMDP, significantly reducing computational overhead without sacrificing overall model performance. Because the update removes the targeted content from the weights instead of masking it, GROM also withstands the low-bit quantization attack that recovers much of the content a gradient-based baseline had appeared to forget. Our code is publicly available at https://github.com/Batorskq/GROM.
