Search papers, labs, and topics across Lattice.
The paper introduces FABLE, a two-stage framework for unstructured model editing that first anchors discrete facts in shallow layers and then updates deeper layers for coherent text generation. This approach addresses the limitations of existing methods that memorize text holistically without fine-grained fact access. Experiments on a new diagnostic benchmark, UnFine, demonstrate that FABLE significantly improves fine-grained question answering while maintaining state-of-the-art holistic editing performance.
Decoupling fact injection from text generation lets you edit LLMs with greater precision, improving fine-grained question answering without sacrificing overall editing performance.
Unstructured model editing aims to update models with real-world text, yet existing methods often memorize text holistically without reliable fine-grained fact access. To address this, we propose FABLE, a hierarchical framework that decouples fine-grained fact injection from holistic text generation. FABLE follows a two-stage, fact-first strategy: discrete facts are anchored in shallow layers, followed by minimal updates to deeper layers to produce coherent text. This decoupling resolves the mismatch between holistic recall and fine-grained fact access, reflecting the unidirectional Transformer flow in which surface-form generation amplifies rather than corrects underlying fact representations. We also introduce UnFine, a diagnostic benchmark with fine-grained question-answer pairs and fact-level metrics for systematic evaluation. Experiments show that FABLE substantially improves fine-grained question answering while maintaining state-of-the-art holistic editing performance. Our code is publicly available at https://github.com/caskcsg/FABLE.
