Track the frontier, from condensed matter to cosmology.
Search papers, labs, and topics across Lattice.
This paper addresses the challenge of efficiently assigning LLMs to specialized roles within agentic workflows by optimizing for multi-objective trade-offs between task quality, deployment cost, and latency. The authors introduce Cost-Coupled Gittins (CC-Gittins), a novel multi-objective search algorithm that leverages the correlation between evaluation and deployment costs to intelligently explore the assignment space. Empirical evaluation demonstrates that CC-Gittins recovers a significant portion of the true Pareto frontier at a fraction of the cost of exhaustive search, offering a practical solution for cost-efficient LLM deployment.
Discover the optimal LLM assignments for agentic workflows with 90% fewer evaluations than brute-force, balancing quality, cost, and latency.
LLM agentic workflows increasingly employ different models for specialized roles. Users may prefer different trade-offs between task quality and deployment cost, sometimes also considering latency. Finding good assignments incurs a separate search cost from evaluating multiple candidates. We introduce Cost-Coupled Gittins (CC-Gittins), a multi-objective extension of single-objective cost-aware Gittins search for Pareto recommendation. It exploits the coupling between evaluation and deployment costs: assignments that cost more to deploy also cost more to evaluate. The search alternates between seeking higher quality and lower deployment cost, reusing each evaluation to inform both searches. After each batch, CC-Gittins recommends assignments that are nondominated under uncertainty-adjusted estimates of quality and deployment cost. We evaluate CC-Gittins on eight quality--cost matrices from workflows with two to five roles and up to 1,681 candidate assignments. We also evaluate the method with latency as a third objective. Across the eight benchmarks, CC-Gittins recovers a substantial fraction of the true Pareto frontier at roughly 10% of the exhaustive-evaluation cost, and recovers most of the frontier with few false positives by 30%.
Microsoft Research