Search papers, labs, and topics across Lattice.
This paper investigates the breakdown in the evidence-to-action chain for tool-using agents, revealing that correct outcomes do not imply evidence-backed decision-making. Through analysis across ten configurations using the SafeActBench benchmark and a provenance-bound Evidence Ledger, the study finds that failures often stem from incomplete investigation or premature action, even when static action assessment is strong. The research highlights that while single-action execution is generally reliable once evidence is gathered, multi-action workflows expose issues with unresolved prerequisites and incomplete execution, indicating failures in how agents utilize established evidence.
Tool-using agents frequently act without sufficient evidence, even when capable of strong static assessment, due to premature execution and incomplete investigation.
Tool-using agents make consequential changes to external state, yet correct outcomes do not guarantee that their actions were supported by evidence established beforehand. We study where this evidence-to-action chain breaks as agents move from deciding whether to act to executing single actions and dependent workflows. Across ten model-harness configurations, strong static action assessment can coexist with much weaker interactive execution. Failures often begin before execution: agents stop with incomplete investigation or act before required evidence is established. Once required evidence is obtained, single-action execution is usually reliable, while multi-action workflows additionally expose unresolved prerequisites and incomplete execution. For this analysis, we introduce SafeActBench, comprising 656 cases across six operational domains and five protocols that progress from static action judgment and investigated non-action to single- and multi-action workflows. A provenance-bound Evidence Ledger and deterministic trajectory evaluator track what information was established, when actions occurred, and whether downstream dependencies were satisfied. These results show that failures arise not only from missing information, but also from how agents use established evidence when deciding and executing actions.
