Search papers, labs, and topics across Lattice.
This study investigates how safety rules are compromised during a single-cycle self-summarization process in long-running AI agents, revealing that presence checks can provide a false sense of security. The authors demonstrate that while some rules may appear intact post-compaction, they often fail to function as intended, leading to significant behavioral violations. Key findings indicate that degraded residues of rules can result in a marked increase in prohibited actions, underscoring the inadequacy of current evaluation methods that rely solely on textual presence.
A seemingly intact safety rule can still lead to significant behavioral violations, as it may not function correctly despite its presence in the model's context.
Long-running agents periodically compact their context, replacing the transcript with a model-generated summary.Recent work shows that dropping a standing safety constraint during compaction drives behavioral violations acrossmany models (Governance Decay; Chen, 2026). We ask a finer question: under a single compaction cycle, how is a safetyrule lost, and what does that imply for detection and evaluation? Our central finding is that a presence check is not asafety check: when compaction does not drop a rule outright, it often leaves something that looks like a rule but doesnot act like one. On behavioral replay, a degraded residue leads the model to perform the prohibited action far more oftenthan an intact welded rule does (all-case gaps of +34 and +57 points under two replay models, both positive), category-level survival behaves like a residue, and even intact rules sometimes fail to fire, so an audit that checks only textualpresence gives false assurance. Sharpening this, rule-form items are retained substantially more often than prominence-matched facts, which is exactly why presence-based checking feels adequate even though survival is not protection.Textual loss is regime-dependent (weld-or-drop with a single rule; degraded predicate-loss residues under a tighterbudget), and we did not observe the hypothesized textual severing mode. Such loss is silent at runtime and detectableonly by comparison with retained external ground truth (such as a constraint registry), which reveals textual absence butnot whether a surviving rule still fires. We also document evaluation pitfalls where LLM-judge labels alone would havereversed a conclusion. All results concern a single compaction cycle.
