Search papers, labs, and topics across Lattice.
This study introduces a foundation model oracle that analyzes git-style diffs to detect behavioral changes in Python refactoring implementations. By evaluating 1,152 refactoring attempts, the researchers identified 13 distinct bugs across seven refactoring types, with a high acceptance rate of reported issues by developers. These findings underscore the critical need for enhancing the reliability of automated refactoring tools to maintain software integrity and developer productivity.
Identifying 13 distinct bugs in Python refactoring implementations reveals significant gaps in current automated tools that could jeopardize software reliability.
Python is a widely adopted programming language, valued for its simplicity and flexibility. However, automated refactoring for Python remains challenging, even though refactoring is an essential practice in software evolution aimed at improving internal code structure without changing external behavior. Understanding how behavioral changes are introduced during refactoring is crucial, as such issues can compromise software reliability and reduce developer productivity. We propose an approach based on a foundation model oracle that analyzes git-style diffs to identify behavioral changes introduced by Python refactorings. We evaluated our technique on Rope refactoring implementations, reusing 1,152 refactoring attempts from a prior study and analyzing 217 resulting transformation pairs with the oracle. Our model-based analysis uncovered 13 distinct bugs among the seven refactoring types studied. All reported bugs were submitted to the respective developers, and 12 of the 13 resulting issue reports were accepted according to issue-tracker evidence. These results highlight the need to improve the robustness of current Python refactoring tools to ensure the correctness of automated code transformations and support reliable software maintenance.
