Search papers, labs, and topics across Lattice.
This paper introduces GRAPHSU, a novel graph-guided selective unlearning method for language models that enhances the deletion scope beyond merely identified forget examples. By constructing a weighted support-route graph and applying graded forgetting strengths, GRAPHSU effectively mitigates knowledge leakage from paraphrases and neighboring examples. The method significantly reduces soft leakage by up to 49.5 percentage points compared to traditional seed-only approaches on benchmarks like TOFU and PISTOL, underscoring the importance of controlling support routes in enterprise unlearning scenarios.
Forgetting in language models just got smarter鈥擥RAPHSU slashes knowledge leakage by nearly 50% by targeting support routes, not just forget seeds.
Enterprises fine-tune language models on proprietary data that may later require removal due to privacy, contractual, or compliance obligations. Selective unlearning removes requested knowledge while preserving model utility, offering a practical alternative to full retraining, but existing methods treat the explicitly identified forget examples as the complete deletion scope. This is insufficient when target knowledge remains recoverable through paraphrases, aliases, or neighboring training examples. We propose GRAPHSU, a graph-guided controller that expands the deletion scope beyond forget seeds by constructing a weighted support-route graph, propagating deletion pressure through it, and applying graded forgetting strengths to high-risk neighbors. On the Task of Fictitious Unlearning (TOFU), a synthetic author-profile question-answering benchmark, and PISTOL, a structural-unlearning benchmark built around interconnected factual samples, with GPT-2 Medium and Llama-3.2-3B-Instruct, GRAPHSU achieves the lowest utility-feasible soft leakage across all deletion settings, reducing leakage by up to 49.5 percentage points over a matched seed-only baseline, demonstrating that effective enterprise unlearning requires controlling support routes, not just forget seeds.
