Search papers, labs, and topics across Lattice.
This paper investigates the reliability of Knowledge Editing (KE) in large language models (LLMs) by revealing that edited information is not fully erased but rather redistributed within the model's representation space. Through a mechanistic analysis, the authors demonstrate that KE methods function as targeted suppression mechanisms, which reduce the likelihood of original facts being expressed without actually removing them. The findings highlight significant vulnerabilities in KE approaches, showing that edited knowledge remains susceptible to adversarial prompting, thereby questioning the efficacy of current KE techniques in practical applications.
Edited knowledge in LLMs is often not erased but merely suppressed, revealing critical vulnerabilities in Knowledge Editing methods.
Knowledge Editing (KE) has emerged as a frontier for updating specific facts in LLMs without costly retraining, but its reliability and underlying mechanisms remain poorly understood. In this work, we examine KE from an adversarial elicitation perspective, revealing that edited knowledge is often not fully erased and continues to surface, with consistent failures observed across diverse model architectures. To explain this behavior, we conduct a mechanistic analysis of popular KE methods. We show that low-rank updates do not overwrite existing knowledge but instead redistribute it within the model's representation space. Furthermore, we find that these methods act as targeted suppression mechanisms that reduce the likelihood of expressing original facts, rather than removing them from the model. Analysis of the loss landscape reveals that edited knowledge lies in narrow, anisotropic regions that are highly sensitive to perturbations, making them highly vulnerable to indirect prompting and adversarial attacks. By exposing these profound architectural vulnerabilities, our work proves that KE algorithms are inherently bypassable and motivates a fundamental reevaluation of how we deploy post-hoc updates in several LLM applications.
