Search papers, labs, and topics across Lattice.
This study investigates the phenomenon of over-privileged tool selection in LLM agents, where agents opt for higher-privilege tools despite the availability of sufficient lower-privilege alternatives. Using the newly developed ToolPrivBench, the authors reveal that such over-selection is prevalent across multiple domains and exacerbated by transient tool failures, highlighting a significant safety concern. The introduction of a privilege-aware post-training defense demonstrates a substantial reduction in unnecessary high-privilege tool use while maintaining overall agent capabilities.
Over-privileged tool selection is alarmingly common in LLM agents, often triggered by transient failures, raising critical safety concerns in autonomous decision-making.
As LLM agents increasingly select tools autonomously, their choices among tools with different privileges become safety-relevant. However, prior tool-selection studies focus on safety-agnostic metadata preferences, leaving privilege-sensitive choices underexplored. To address this gap, we study over-privileged tool selection, in which an agent selects or escalates to a higher-privilege tool despite a sufficient lower-privilege alternative. We introduce ToolPrivBench to evaluate whether agents choose higher-privilege tools despite sufficient lower-privilege alternatives, measuring both initial selection and escalation after transient tool failures. Across eight domains and five recurring risk patterns, we find that over-privileged tool selection is common among mainstream LLM agents and is further amplified by transient failures. We further find that general safety alignment does not reliably transfer to least-privilege tool choice, while prompt-level controls provide only limited mitigation under transient failures. We therefore introduce a privilege-aware post-training defense that teaches agents to prefer sufficient lower-privilege tools and escalate only when necessary. Our mitigation experiments show that this defense substantially reduces unnecessary high-privilege tool use while preserving general capabilities.
