Search papers, labs, and topics across Lattice.
This paper addresses the challenge of backdoor attacks in large language models (LLMs) by introducing a localized detoxification method that avoids full model retraining. The authors utilize activation patching and curvature analysis to identify and target the specific modules responsible for trigger-induced behavior, applying low-rank repair to these modules. Experimental results demonstrate that their approach effectively mitigates malicious responses while maintaining the integrity of benign outputs, highlighting a new perspective on backdoor removal as a structural repair issue.
Targeted low-rank repair can effectively detoxify backdoored LLMs without the need for full retraining, preserving benign behavior while suppressing malicious outputs.
Backdoor attacks pose a serious threat to large language models (LLMs) by causing otherwise benign systems to produce attacker-specified malicious behavior when a hidden trigger is present. In this work, we study post hoc detoxification of backdoored LLMs in a practical setting where the defender has access to the poisoned model but does not wish to retrain the full network from scratch. We propose a mechanistically guided weight-space repair framework that first localizes modules involved in propagating trigger-induced behavior using activation patching and Fisher/K-FAC curvature analysis, and then applies targeted low-rank repair to only the most influential modules. We evaluate the method on poisoned variants of \texttt{Llama-3.2-1B-Instruct} with triggers inserted at the beginning, middle, and end of otherwise benign prompts. Results show that the proposed approach substantially suppresses trigger-conditioned malicious responses while preserving benign model behavior. These findings suggest that backdoor removal in LLMs can be formulated as a localized structural repair problem rather than only a broad behavioral alignment problem.
