Search papers, labs, and topics across Lattice.
This study investigates the impact of knowledge poisoning on Retrieval-Augmented Generation (RAG) systems used by AI security agents, revealing that a single poisoned write-up can systematically alter agent behavior across various challenges and vulnerabilities. The authors introduce the concept of the Verification Boundary (VB), which classifies the agent's ability to refute claims based on available evidence, highlighting the systematic nature of poison adoption. Key findings indicate that while verification prompting and multi-source retrieval improve performance with strong evidence, they falter in scenarios with sparse evidence or zero-day vulnerabilities.
A single poisoned write-up can systematically mislead AI security agents, fundamentally altering their exploit behavior across real-world challenges.
AI security agents increasingly rely on Retrieval-Augmented Generation (RAG) to use external security knowledge for vulnerability analysis and exploit reasoning. This creates a new risk: poisoned write-ups can be operationalized into incorrect exploit behavior. Yet, prior work on RAG poisoning has mostly studied answer corruption in QA settings, much less is known about action-taking security agents. This paper aims to reveal such characteristics with crafted poisons about real-world challenges and AI agents. First, we demonstrate how a crafted single poisoned write-up injected into public-style security knowledge sources which we denote as Poisoned Playbooks, alters the behavior of RAG-based AI security agents. Across 11 CTF challenges, 3 frontier LLM families, 2 model generations, and 11 real-world CVEs, we find that poison adoption is systematic rather than random. To explain this pattern, we introduce the Verification Boundary (VB), a 3-level empirical classification based on what evidence the agent can use to refute a retrieved claim. Finally, we evaluate verification prompting and multi-source retrieval, showing that both help when stronger evidence exists, but weaken under sparse-evidence and zero-day conditions.
