Search papers, labs, and topics across Lattice.
TrajAudit is introduced as a novel framework for diagnosing failures in agentic coding systems, specifically addressing the challenges of noisy and long execution trajectories in repository-level tasks. It uses an investigator agent with modules for filtering irrelevant information and generating preliminary diagnoses from test failures. Evaluated on the new RootSE benchmark, TrajAudit achieves a 24.4% improvement in localization accuracy and reduces token consumption by 18% compared to existing methods.
Debugging agentic coding systems just got a whole lot easier: TrajAudit pinpoints failures with significantly higher accuracy and lower token cost than existing methods.
Agentic systems have been widely studied to automate software engineering jobs such as bug fixing. As these systems increasingly tackle complex tasks, understanding where and why they fail becomes essential for iterative refinement and operational reliability. Existing automated failure diagnosis approaches leverage task execution trajectories, yet their effectiveness degrades substantially as trajectory length and complexity increase. For repository-level coding tasks specifically, trajectories are laden with noise, such as redundant program structure and verbose code context. Moreover, these trajectories are very long, while long-context reasoning remains a known weakness of LLMs. To address these two challenges, we propose TrajAudit, the first failure diagnosis framework for repository-level coding trajectories. TrajAudit employs an investigator agent supported by two modules: one filters failure-irrelevant information through pattern matching and keyword detection, and the other generates a preliminary diagnosis from test failure reports as prior knowledge, helping the agent handle noisy long contexts. The investigator agent can further invoke tools to retrieve filtered content on demand, ensuring that critical information is preserved while noise is minimized. We also introduce RootSE, a benchmark of 93 real-world agentic failure instances sourced from software maintenance tasks, representing the most complex trajectory diagnosis benchmark to date. Experiments on RootSE show that TrajAudit outperforms all existing baselines by over 24.4 percentage points in localization accuracy, while reducing token consumption by at least 18%, demonstrating its practical effectiveness. We hope this work draws community attention to failure management in agentic software engineering and provides a foundational resource for future research.
