Search papers, labs, and topics across Lattice.
This paper addresses the challenge of tool selection in LLM agents by introducing a cost-aware marginal decision-focused stopping (CAM-DF) framework, which optimally balances the trade-off between acquiring sufficient tools and minimizing costs and privacy risks. The authors demonstrate that existing methods fail to account for heterogeneous costs, leading to suboptimal tool acquisition decisions. Evaluating CAM-DF across 1,343 tasks, they show it outperforms traditional methods, achieving higher payoffs while reducing tool exposure by 37% during live execution.
CAM-DF reduces tool acquisition costs by 37% while maintaining task success, challenging the conventional wisdom that more tools always lead to better outcomes.
As LLM agents increasingly depend on diverse external services such as search engines, databases, and connectors, agent harnesses face a fundamental tool-selection challenge: acquiring too few tools leaves the task under-informed, while too many adds cost, context load, and privacy exposure. Routers and retrievers can rank candidate tools by relevance, but a ranking alone does not determine how many are worth selecting. Existing approaches leave acquisition under heterogeneous costs unaddressed. We formulate this decision as cost-aware marginal decision-focused stopping (CAM-DF) over ranked tool prefixes, with CAM-DF-lite as a compact interpretable variant. We train directly on the offline gap between stopping now and the best continuation: its sign labels the decision, its magnitude weights each error by the payoff at stake. We prove this objective is Bayes-aligned with the stopping target and that score-only rules are suboptimal under heterogeneous costs. We evaluate on 1,343 tasks across five tool-use domains. On $蟿$-bench Retail, CAM-DF attains the highest payoff among deployable methods, with gains over a predict-then-threshold baseline across all five ranking sources and two cost regimes. Our approach is state-of-the-art under heterogeneous costs and high cost pressure, with larger gains under weaker rankings. In live execution, CAM-DF exposes the agent to 37\% fewer tools than full access while maintaining comparable task success. The CAM-DF family is a lightweight pre-execution plugin that turns existing tool rankings into lower-cost acquisition decisions without fine-tuning the underlying LLM.
Tsinghua AI