Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper evaluates context attribution methods for large language models (LLMs) and reveals their limitations when distinguishing between in-context contributions and those derived from the model's training weights. By introducing a novel evaluation protocol with four metrics and a benchmark dataset, the authors systematically assess the reliability of attribution scores in scenarios where context overlaps with training data. The findings indicate that existing methods yield unfaithful attributions, particularly when knowledge is shared between context and model weights, underscoring the need for improved disentanglement techniques.
Context attribution methods fail to reliably distinguish between in-context and in-weight contributions, leading to unfaithful scores that could misguide model interpretation.
Context attribution methods for large language models (LLMs) identify which input context contributes to the model response. Recent works show the initial success in attributing the con- tributive score of the contexts. However, we observe that when the context overlaps with the training data, these methods can- not disentangle in-context from in-weight (IW) contributions, producing unreliable scores. Based on this observation, in this work, we introduce: 1) an evaluation protocol that relies on four new metrics (base-model context attribution score (BCS), cross-model context attribution consistency (CAC), attribution preservation score (APS), source separation pre- cision (SSP)) and 2) a benchmark dataset (WMDP-Cyber++) with ground-truth provenance labels to systematically assess attribution under IW overlap. In our experiments across four well-known context attribution methods, we demonstrate that they provide unfaithful attribution when the knowledge from the context also exists in the weights. Finally, we adapt these methods for source separation (IW vs. in-context learning (ICL)) and show that they cannot do the disentanglement based on the contributive score