Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper presents an empirical study of bug fix patterns in LLM agents, analyzing data from Stack Overflow, GitHub, and HuggingFace Forums to understand common errors and fixes. They introduce AgentDefect, a new benchmark dataset comprising 37 runtime buggy instances with corresponding fixes and tests. Based on these findings, they propose SelfHeal, a multi-agent system employing two ReAct agents (fix and critic) that leverage internal fix rules and external web search to automatically repair bugs in LLM agents, achieving state-of-the-art performance with Gemini 3 Pro.
LLM agents are riddled with predictable bugs, and a multi-agent system using fix rules and web search can automatically patch them, outperforming existing methods.
Large Language Models (LLMs) have transformed software development and AI applications. While LLMs are designed for text processing, LLM agents extend this capability by enabling autonomous actions, tool use, and multi-step task completion. As this field grows, developers face new challenges in debugging these complex systems. To address this challenge, we present the first empirical study on bug fix patterns in LLM agents. We study buggy posts and code snippets from three platforms: Stack Overflow, GitHub, and HuggingFace Forums. We examine their fix patterns, the components where fixes are applied, and the programming languages and frameworks involved. Furthermore, we introduce AgentDefect, the first benchmark dataset for bugs in LLM agents. The dataset contains 37 runtime buggy instances along with fixed code and test files. Finally, we present SelfHeal, a multi-agent system designed to fix bugs in LLM agents. The system leverages two independent ReAct agents: the fix agent and the critic agent. These agents use tools that provide both internal knowledge (fix rules) and external knowledge (web search) to propose and validate fixes. Our evaluation shows that SelfHeal with Gemini 3 Pro as the backbone LLM outperforms both baseline and state-of-the-art approaches by a significant margin.