Search papers, labs, and topics across Lattice.
This study evaluates the effectiveness of specialist agentic systems versus generalist large language models (LLMs) in executing structured workflows derived from Business Process Model and Notation (BPMN) diagrams. By focusing on deterministic workflows, the researchers developed a specialized workflow agent that significantly outperformed generalist agents like Roo and Cline in terms of tool-use exactness, latency, and error rates. The findings indicate that specialist agents not only enhance performance metrics but also drastically reduce costs and repair iterations, highlighting their potential for reliable industrial applications.
Specialist agents can outperform generalist LLMs by up to 20 percentage points in execution accuracy while slashing costs and errors, making them essential for reliable software development.
Large Language Models (LLMs) have accelerated the adoption of software development agents, now widely available as Integrated Development Environment (IDE) extensions and standalone applications. While these agents are typically general-purpose, it remains unclear whether specialist agents justify their additional development effort. We investigate this question in the context of business process automation, focusing on the transformation of Business Process Model and Notation (BPMN) diagrams into executable agentic workflows. Since BPMN specifies explicit control-flow semantics, we focus on deterministic workflows in which a fixed process model and inputs uniquely determine the executed path. We introduce a specialist workflow for this task and compare it against generalist agents such as Roo and Cline. Our results show that the specialist solution produces agents that outperform generalist baselines by approximately 9-20 percentage points in tool-use exactness, 2-4x in penalty-adjusted latency, and 3x fewer tool-call errors, while reducing generation token cost by over 95% and eliminating repair iterations. We also find that generalist agents generate code inconsistently in both functionality and quality, limiting their suitability for industrial settings where reliability and maintainability are essential.
