Search papers, labs, and topics across Lattice.
This paper introduces RefactorAssist, an agentic approach designed to enhance the reliability of code refactoring generated by Large Language Models (LLMs). By analyzing failures in LLM-generated refactorings across 10 open-source Java projects, the authors identify key failure modes, including context misunderstanding and incorrect renaming, and implement a multi-step repair process that combines static checks with test-guided refinements. The results demonstrate a significant improvement in functional correctness, achieving a 94.2% cumulative pass rate for refactorings under optimal conditions, thereby facilitating more practical adoption of LLMs in software engineering tasks.
LLM-generated code refactorings can achieve a 94.2% pass rate with the right agentic refinements, addressing critical failure modes that have hindered their practical use.
Code refactoring aims to enhance the internal structure of source code without affecting its functional behavior. The recent advancements of Large Language Models (LLMs) have demonstrated potential for automating software engineering tasks, such as code refactoring. However, the refactorings produced by LLMs often introduce subtle errors, leading to functional behavior changes and failed unit tests, which limit their practical adoption. To address the limitations of LLM-generated refactorings, we analyze the root causes of their failures and develop the RefactorAssist agent to improve the functional correctness of LLM-generated refactorings. To this end, we use 10 open-source Java projects with their native test suites and manually evaluate why LLM-generated refactorings fail unit tests. We then design an agentic approach that leverages unit-test logs, error explanations, project context retrieval, and code diffs to guide the iterative refactoring. Our findings show that the main reasons for failure are context misunderstanding/hallucination (24.3%), incorrect or inconsistent renaming (15.3%), adding new functionality or variables (13.7%), code incompleteness (11.3%), syntax and structural errors (9.7%), edge cases not handled (9%), improper type handling (8.7%), and variables outside scope (8%). To make our approach cost-effective, RefactorAssist first applies a static repair step for missing imports, unbalanced brackets, and compilation errors without LLMs. For remaining failures, RefactorAssist incorporates error logs and code diffs, achieving up to a 70.8% repair rate on the remaining failures and a 94.2% cumulative pass rate under the best-performing configuration. These results indicate that static checks and test-guided, context-aware agentic repair can increase the reliability of LLM-generated refactorings, bringing them closer to practical integration within developer workflows.
