Song Wang

A large-scale empirical study of quality assurance (QA) practices in 157 open-source LLM-based agent projects with at least 100 GitHub stars highlights the need to move beyond feature-level testing toward systematic end-to-end validation that ensures agent workflows remain within intended boundaries when interacting with untrusted inputs, tools, persistent state, and external APIs.

AgentGuard, an instruction-level guardrail framework that learns conditional execution constraints from anomalous trajectories of coding agents, is presented, demonstrating that execution guardrails learned from historical failures can substantially improve the reliability of AI coding agents while highlighting the remaining challenge of balancing safety and task completion.

LLM-powered program repair tools can memorize bug fixes, but struggle with minor syntactic changes and applying fixes in real-world projects, revealing a critical gap in their reasoning abilities.