Search papers, labs, and topics across Lattice.
This paper identifies a critical flaw in stateful language agents where retained key/value (KV) cache can lead to inconsistencies during logical rollbacks, allowing models to attend to previously discarded content. The authors formalize this issue as a lack of rollback consistency and demonstrate that a correct logical rollback does not necessarily restore the integrity of the retained inference state. Through a novel same-token/different-cache audit across various model families, they reveal that retained KV can significantly alter model behavior, with 25 out of 63 audited cells showing unexpected effects, while rebuilding the cache resolves these discrepancies.
Retained KV cache can undermine rollback consistency in language agents, leading to unexpected model behavior even after logical aborts.
Stateful language agents assume a rejected branch can be taken back by clearing it from the application transcript. We show this breaks when the serving session retains key/value (KV) state across the logical abort: the model can continue attending to content the application believes it discarded. We formalize the missing guarantee as rollback consistency: a complete abort must restore the state the model attends, not just the transcript. The key failure is cross-layer: a correct logical rollback need not compose with retained inference state, and the gap can remain invisible to the application. To isolate cache effects from text effects, we introduce a same-token/different-cache audit that holds decision-step tokens identical while varying only whether the cached prefix is stale or rebuilt from committed state. Across seven open-weight families (3.8B-36B), retained KV alone flips a typed protected effect in 25 of 63 audited cells, while attacker tokens are absent from the served request in all 63; rebuilding the cache closes every cell. The channel reproduces in an end-to-end session application, on the default Hugging Face Transformers cache-reuse path, and under LangGraph time-travel, where verified logical rollback can still leave attended KV stale. Susceptibility varies across models, but the underlying attended-state integrity violation is structural. We rule out position and length confounds, generalize across protected effects, policy structures, and a cache-isolated Mixture-of-Experts model, and show that transaction-local cache restoration closes the channel without requiring a global cache flush. All headline results are deterministic and reproducible from released artifacts.
