Search papers, labs, and topics across Lattice.
Agentic CLEAR is introduced as an automated evaluation framework for LLM agents, addressing the limitations of current tools by providing dynamic, multi-level insights into agent behavior. It analyzes agent performance at system, trace, and node levels, generating textual feedback based on observed behavior. Experiments across diverse benchmarks demonstrate Agentic CLEAR's ability to produce high-quality feedback, aligning with human annotations and predicting task success.
Current LLM agent evaluation tools are stuck in the Stone Age, but Agentic CLEAR automates dynamic, multi-level analysis, finally offering insights that adapt to the rapidly evolving agent landscape.
Agentic systems are becoming more capable: agents define strategies, take actions, and interact with different environments. This autonomy poses serious challenges for overseeing and assessing agent behavior. Most current tools are limited, focusing on observability with basic evaluation capabilities or imposing static, hand-crafted error taxonomies that cannot adapt to new domains. To address this gap, we present Agentic CLEAR, an automatic, dynamic, and easy-to-use evaluation framework. It produces textual insights into the agent behavior on three levels of granularity: system, trace, and node. Agentic CLEAR operates above the observability layer, enabling seamless integration and featuring an intuitive UI that makes agent evaluation highly accessible. In our experiments on four benchmarks, seven agentic settings, and tens of thousands of LLM calls, we show that Agentic CLEAR produces high-quality, data-driven, insightful feedback. Our analysis shows strong alignment with human-annotated errors and the ability to predict task success rate.
