Search papers, labs, and topics across Lattice.
The paper introduces HiEdit, a hierarchical reinforcement learning framework for lifelong model editing that dynamically selects the most relevant layers for each edit. HiEdit addresses the limitations of static, dense parameter perturbations in existing LME methods, which can lead to catastrophic forgetting. By using a hierarchical RL agent to identify and selectively update knowledge-relevant layers, and incorporating a sparsity reward, HiEdit achieves more precise and localized updates. Experiments demonstrate that HiEdit outperforms RLEdit by 8.48% while perturbing only half the layers, showing improved adaptability and reduced forgetting.
Forget monolithic updates: HiEdit's hierarchical RL finds that editing specific knowledge in LLMs is best done by surgically perturbing only the relevant layers, boosting performance and cutting compute by half.
Lifelong model editing (LME) aims to sequentially rectify outdated or inaccurate knowledge in deployed LLMs while minimizing side effects on unrelated inputs. However, existing approaches typically apply parameter perturbations to a static and dense set of LLM layers for all editing instances. This practice is counter-intuitive, as we hypothesize that different pieces of knowledge are stored in distinct layers of the model. Neglecting this layer-wise specificity can impede adaptability in integrating new knowledge and result in catastrophic forgetting for both general and previously edited knowledge. To address this, we propose HiEdit, a hierarchical reinforcement learning framework that adaptively identifies the most knowledge-relevant layers for each editing instance. By enabling dynamic, instance-aware layer selection and incorporating an intrinsic reward for sparsity, HiEdit achieves precise, localized updates. Experiments on various LLMs show that HiEdit boosts the performance of the competitive RLEdit by an average of 8.48% with perturbing only half of the layers per edit. Our code is available at: https://github.com/yangfanww/hiedit.
