Search papers, labs, and topics across Lattice.
This paper introduces InjecMEM, a memory injection attack that exploits vulnerabilities in the memory systems of LLM agents by requiring only a single interaction to manipulate future responses. The method employs a retrieval-then-generate mechanism, utilizing a retriever-agnostic anchor and an adversarial command to ensure consistent association with the target topic while remaining effective across various contexts. Evaluations demonstrate that InjecMEM can reliably achieve topic-conditioned retrieval and targeted generation, highlighting significant security concerns in LLM memory systems and the necessity for enhanced defenses.
A single interaction is all it takes for an adversary to manipulate LLM memory systems, steering outputs toward a pre-specified target with alarming precision.
Memory is becoming a default subsystem in deployed LLM agents to provide persistent personalization and continuity. This naturally prompts a question: will memory system introduce new vulnerabilities into agents? Thus we propose InjecMEM, a novel memory injection attack paradigm that requires only a single interaction (no read/edit access to memory store) to steer later responses of related queries toward a pre-specified output. Guided by the retrieval-then-generate mechanism of memory systems, we craft the injection with a retriever-agnostic anchor and an adversarial command. The anchor contains high-recall topical cues so that downstream retrieval consistently associates the record with the target topic. The command is a short sequence optimized to remain effective under uncertain fused contexts, variable placements, and long prompts so that it reliably steers outputs once retrieved. We learn the command via gradient-based coordinate search, averaging over synthetic prompt templates and insertion positions, and extend it to joint optimization across backbones to study transfer. Evaluated across multiple memory systems and backbone models, InjecMEM achieves reliable topic-conditioned retrieval and targeted generation, remains effective under memory drift, and leaves non-target queries unaffected. Our results underscore the need to harden memory systems and provide a reproducible framework for studying agent memory.
