Search papers, labs, and topics across Lattice.
This study empirically investigates the attention patterns of Large Language Models (LLMs) in the context of Automated Program Repair (APR) by analyzing 319 real-world bugs in Python and Java. The findings reveal that successful repairs are linked to a more diffuse attention across various components of bug reports, while failures tend to focus too narrowly on metadata. Importantly, the research establishes a correlation between the alignment of model attention with developer-identified critical sections and higher repair success rates, highlighting attention misallocation as a significant factor in APR performance inconsistencies.
Attention misallocation in LLMs is a critical factor behind their inconsistent performance in Automated Program Repair, with successful outcomes linked to broader attention across bug report components.
Large Language Model (LLM)-based Automated Program Repair systems are advancing rapidly, yet their performance remains inconsistent. Even when provided with the same contextual information, an LLM may generate a correct patch for one bug but fail on another closely related bug. Why this happens remains poorly understood, and it is unclear how LLMs prioritize the diverse information in bug reports and whether model attention affects repair success. In this paper, we present the first empirical study of attention patterns in LLM-based program repair, providing interpretable insights into how models process bug reports and where their attention is concentrated during repair. We analyze 319 real-world Python and Java bugs from SWE-bench Verified and Multi-SWE-bench to study (RQ1) how model attention is distributed across bug report sections, (RQ2) how attention patterns within each section differ between successful and unsuccessful repairs, and (RQ3) how these patterns compare to information developers consider important for bug fixing. We find that successful repairs are characterized by diffused attention across multiple diagnostic components such as bug descriptions, stacktraces, and test cases, while failures often exhibit over-localized attention toward metadata such as version information. We further observe that stronger alignment between model attention and developer-identified key sections and phrases is associated with higher repair success. Our results provide the first empirical evidence that attention misallocation is a key factor in LLM-based APR failures, and offer actionable insights for designing more interpretable and reliable future APR systems.
