Search papers, labs, and topics across Lattice.
This paper identifies a critical flaw in the safety mechanisms of autonomous large language model (LLM) agents, revealing that existing trajectory-scoped monitors fail to effectively detect threats that span multiple iterations. The authors demonstrate that these monitors have a true-positive rate equal to their false-positive rate when faced with fragmented evidence, thus compromising safety. They introduce LoopHarness, a novel approach that maintains a persistent, non-decaying safety state across iterations, significantly improving the detection of unauthorized actions in autonomous LLM operations.
Trajectory-scoped safety monitors for LLM agents are fundamentally flawed, leading to a 50% chance of missing threats that span multiple iterations.
Large language model agents are increasingly deployed as autonomous loops. Starting from one human goal, such a system repeatedly discovers work, plans, executes tool calls, verifies outcomes and persists state across many unattended iterations. The agent safeguards in wide use, however, are defined over a single trajectory, and their safety state is re-initialized when the next trajectory begins. We show that this is a failure of composition rather than an implementation detail. Our central result is a separation: against an attack whose evidence is fragmented across several iterations, every trajectory-scoped monitor has a true-positive rate equal to its false-positive rate, however expressive it is, because the evidence it would need never appears in the window it sees, whereas a monitor retaining cross-iteration state separates the two perfectly. We further show that the obvious repair of carrying a geometrically decaying risk score is insufficient, because the cooling-off period a patient adversary must wait is a constant that does not grow with the horizon $N$. We then present LoopHarness, which restores a persistent, non-decaying safety state at the loop level. Under mediated commits and an arbiter detection floor $\delta_M$, it bounds the expected number of unauthorized irreversible actions by $B+m-1+m/\delta_M$, a constant in $N$, of which the $B+m-1$ term is decided by a model-free rule and therefore survives a fully colluding verifier. We give a complete evaluation protocol on native Agent-SafetyBench tasks with paired clean and attacked episodes, an outer-state attack suite whose decisive evidence exists only across iterations, per-module ablations, and an adaptive white-box red team.
Tsinghua AI