Search papers, labs, and topics across Lattice.
This paper identifies and analyzes "structural alignment bias" in LLMs' tool use, where models invoke irrelevant tools simply because query attributes can be assigned to tool parameters. They introduce SABEval, a dataset that decouples structural alignment from semantic relevance, to demonstrate this bias leads to significant tool-invocation errors. Through Contrastive Attention Attribution, they reveal competing pathways for semantic checking and structural matching, and propose a rebalancing strategy to mitigate the bias without harming general tool use.
LLMs often invoke irrelevant tools just because the query structure *fits* the tool's parameters, revealing a surprising and widespread flaw in their reasoning.
Large language models (LLMs) have demonstrated impressive capabilities in utilizing external tools. In practice, however, LLMs are often exposed to tools that are irrelevant to the user's query, in which case the desired behavior is to refrain from invocations. In this work, we identify a widespread yet overlooked mechanistic flaw in tool refusal, which we term structural alignment bias: Even when a tool fails to serve the user's goal, LLMs still tend to invoke it whenever query attributes can be validly assigned to tool parameters. To systematically study this bias, we introduce SABEval, a new dataset that decouples structural alignment from semantic relevance. Our analysis shows that structural alignment bias induces severe tool-invocation errors in LLMs, yet remains largely unaccounted for in existing evaluations. To investigate the internal mechanisms underlying this bias, we propose Contrastive Attention Attribution, which reveals two competing pathways for semantic checking and structural matching. The relative strength of these pathways drives LLMs' tool invocation decisions. Based on these findings, we further introduce a rebalancing strategy that effectively mitigates structural alignment bias, as demonstrated by extensive experiments, without degrading general tool-use capabilities.
