Search papers, labs, and topics across Lattice.
This paper introduces AgentEval, a black-box testing framework designed to identify and stress state-dependent failures in conversational LLM agents by mining their conversational workflow graphs. By leveraging the structure of these graphs, AgentEval effectively enumerates specific guards and prerequisites, allowing it to target critical boundaries that standard testing methods often miss. The framework outperforms a white-box auditor by successfully generating tests that cover 23-38 distinct boundaries per agent, demonstrating the importance of structured interaction in revealing hidden vulnerabilities.
AgentEval uncovers up to 38 hidden failure boundaries in conversational LLMs that traditional testing methods overlook.
Conversational LLM agents can cause real-world harm when their internal workflows fail, such as completing a transaction without confirmation. Testing these state-dependent failures is difficult because critical boundaries, such as identity checks and confirmation gates, are hidden behind multi-turn conversational prerequisites, rendering them inaccessible to standard tests. We present AgentEval, a black-box testing framework that discovers and stresses these stateful boundaries. AgentEval interacts with an agent to mine a \emph{conversational workflow graph}, a model of its behavior. Instead of prompting blindly, AgentEval uses this graph's structure to enumerate specific guards and prerequisites as test targets, replaying the conversational path to a boundary before applying a perturbation. AgentEval then executes each test, determining whether it passes or fails using only the conversation turns. We benchmark AgentEval against a privileged, white-box auditor with access to the agent's underlying source code, which AgentEval never sees. On four $\tau^3$-bench agents, AgentEval successfully generates tests covering $23$--$38$ distinct boundaries per agent; ablation studies attribute the gain to the graph's structure: $23$ distinct boundaries versus $12$ with a prompt-only baseline, at lower duplicate and false-alarm rates.
