Search papers, labs, and topics across Lattice.
This paper introduces SmellBench, a novel benchmark designed to evaluate code agents on their ability to refactor code while addressing long-term maintainability issues, rather than just functional correctness. By incorporating 294 cases with various code smells from real-world repositories, the authors provide a controlled environment to assess agents' performance across multiple dimensions, including functional correctness and refactoring quality. Experimental results reveal that even the best-performing agents struggle with a mere 50.34 score in eliminating code smells, highlighting significant limitations in their understanding of code structure and inter-file relationships.
Code agents are only eliminating 50% of code smells, revealing critical gaps in their understanding of maintainability and cross-file dependencies.
Code Agents have achieved remarkable advances in recent years, exhibiting strong capabilities across a wide range of software engineering tasks. However, their misuse often produces bloated and disorganized code that impairing readability, extensibility, and robustness. Despite this risk, existing benchmarks largely evaluate functional correctness rather than long-term maintainability of code agents. In this paper, we propose SmellBench, an extensible code refactoring benchmark that proactively injects code smells into clean code snippets from real-world repositories. This design enables the generation of controlled, high-quality, and diverse refactoring cases with human-written ground truth. Specifically, it contains 294 cases spanning 7 popular smell types, 3 difficulty levels, 2 instruction settings across 7 real-world repositories. We further design 3 evaluation aspects covering functional correctness, localization ability, and refactoring quality assessment. Experiments with 2 popular agents and 6 large langauge models (LLMs) show that the best combination - Qwen Code + Claude Sonnet 4.5 - achieved only a 50.34 score of smell elimination. Further analysis reveals that this gap arises from a focus on local code smells and a lack of cross-file understanding, which hinders comprehensive smell elimination.
