Search papers, labs, and topics across Lattice.
This paper investigates self-state attacks on self-hosted AI agents, where an agent's own memory and configuration files can be compromised through legitimate OS system calls. By characterizing a four-axis attack space and evaluating various defense strategies, the authors reveal that while a layered defense stack can effectively mitigate most attacks, a small residual attack surface remains undetectable at the OS level. These findings highlight the need for a reevaluation of OS defenses against this emerging class of threats, suggesting new avenues for research in AI safety.
A layered defense can thwart most self-state attacks on AI agents, but a hidden vulnerability persists that challenges current OS security paradigms.
Self-hosted AI agents read and write their own memory and configuration files to function. An agent may get compromised via corruption of its own state -- a compromise realized via legitimate OS system call invocation. We refer to this class of threats as self-state attacks. In this paper, we investigate the OS resilience to this class of attacks. Formally, we characterize a four-axis attack space (Target, Mechanism, Granularity, Temporal); investigate the structural limits of prevention, detection, and recovery; and introduce a workload-conditioned view of detectability. To instantiate the framework, we collect live activity traces from a representative self-hosted agent running across distinct workload profiles, and realize the attack space as a 23-cell matrix, 43 concrete operations on real self-state files, and injected into those traces. We then evaluate both canonical and workload-conditioned defense strategies. The empirical results show that a layered defense stack (access-control prevention on the instruction and configuration layers, workload-conditioned detection on the memory layer, and periodic backup for recovery) is effective on most attack cells while a small residual attack surface remains structurally indistinguishable at the OS level. These findings suggest that against the newly established class of self-state attacks, OS-level defense needs to be reconsidered, potentially opening new research directions in the field.
