Search papers, labs, and topics across Lattice.
This paper identifies and explores a new category of indirect prompt injection attacks known as agent data injection (ADI) attacks, where malicious data is disguised as trusted information, leading AI agents to execute unintended actions. The authors demonstrate that ADI attacks can exploit vulnerabilities in real-world AI agents, including arbitrary click attacks and remote code execution, revealing significant security gaps in current agent designs. Their findings indicate that existing defenses against instruction injection attacks are ineffective against ADI, highlighting the urgent need for improved security measures that distinguish between trusted and untrusted data.
Agent data injection attacks can exploit AI agents' vulnerabilities, leading to severe security breaches that existing defenses fail to address.
AI agents act on behalf of user prompts, consuming external data and taking actions based on the agent context. Prior research on AI agent security has primarily focused on indirect prompt injection (IPI). Its most well-studied category is instruction injection, where attacker-controlled untrusted data is interpreted as an instruction. In response, many mitigations have been proposed to prevent instruction injection attacks. In this paper, we introduce a new category of IPI, agent data injection attacks (ADI). ADI injects malicious data disguised as trusted data, such as security-critical metadata (e.g., resource identifiers or data origins) or agent context data (e.g., tool call and response formats). As a result, agents unknowingly execute unintended actions based on attacker-controlled data. ADI has similar attack impacts as instruction injection attacks, because it causes agents to misbehave and execute unintended actions. Despite the similar impact, ADI remains underexplored and easily bypasses existing IPI defenses. We found several critical vulnerabilities in real-world agents that allow an attacker to launch various attacks: arbitrary click attacks on web agents (Claude in Chrome, Antigravity, and Nanobrowser), and remote code execution and supply-chain attacks on coding agents (Claude Code, Codex, and Gemini CLI). We evaluate ADI vulnerabilities across off-the-shelf models and AI agents, and find that ADI is effective in both standalone LLMs and AI agent settings. ADI exposes a critical gap in agent security, signifying that current AI agents do not employ a fundamental security principle: current agents do not isolate trusted data from untrusted data.
