Search papers, labs, and topics across Lattice.
This paper introduces PerspectiveGap, a benchmark designed to evaluate the ability of large language models (LLMs) to effectively compose orchestration prompts for multi-agent systems. By analyzing 110 scenarios through role-fragment assignments and free-form prompt writing, the study reveals that while GPT-5.5 excels in orchestration prompting, achieving a 62.0% pass rate, other models like Opus 4.7 struggle significantly despite strong performance in coding tasks. The findings underscore the challenges in multi-agent orchestration, with an overall average pass rate of only 14.9% across evaluated models, highlighting the need for focused evaluation in this area.
Multi-agent orchestration prompting is critically under-evaluated, with only 14.9% of models passing the new PerspectiveGap benchmark.
Real-world LLM applications are moving beyond single-agent workflows toward orchestrated multi-agent systems, yet current models still struggle to determine what each sub-agent needs to know. To measure this, we introduce PerspectiveGap, a benchmark for evaluating LLMs' ability to compose orchestration prompts for multi-agent systems. PerspectiveGap contains 110 scenarios, each evaluated through two distractor-mixed task formats: role-fragment assignment and free-form prompt writing. These scenarios are organized into 10 topologies, which are distilled from the authors' real-world engineering practice and framed by the Prompt Economy principle: building loop-centered orchestrations that maximize utility with minimal role and engineering overhead. In experiments with 27 commercial models from 10 companies, GPT-5.5 substantially outperforms all competitors, whereas Opus 4.7 shows a notable weakness in orchestration prompting despite its strong coding performance. Nevertheless, PerspectiveGap remains challenging: the evaluated models achieve an average combined pass rate of only 14.9\% (GPT-5.5 62.0\%) and an average overall leakage rate of 246.5\% (a per-scenario information leak-event count, not a proportion; GPT-5.5 49.1\%). These findings suggest that multi-agent orchestration prompting is a distinct and under-evaluated capability, and PerspectiveGap provides a foundation for measuring and improving it systematically.
Stanford HAI