Search papers, labs, and topics across Lattice.
This paper tackles the challenge of Indirect Prompt Injection (IPI) attacks on Large Language Models (LLMs), which compromise their safe deployment by exploiting the indistinguishability between instructions and data. The authors introduce AEGIS, an innovative defense mechanism that utilizes instruction-sensitive projectors and a Unified Multi-Layer Consensus approach to effectively detect and mitigate IPI attacks without incurring significant latency or over-refusal. Empirical results demonstrate that AEGIS significantly outperforms existing defenses, marking a substantial advancement in the safety of LLM applications in complex ecosystems.
AEGIS can effectively defend against Indirect Prompt Injection attacks while maintaining low latency and high utility, a breakthrough for LLM safety.
Large Language Models (LLMs) have been integrated into complex ecosystems (e.g., Code Agents), while Indirect Prompt Injection (IPI) attacks have emerged as critical barriers to their safe deployment. Attackers exploit LLMs' indistinguishability between "instructions" and "data" to manipulate LLMs via maliciously injected instructions. Existing defenses, however, face an intractable safety-utility trade-off: most guardrails either incur high latency or suffer from severe over-refusal. In this paper, we first demonstrate that LLMs can separate instruction from data intrinsically with both theoretical and empirical evidence. Inspired by this insight, we propose AEGIS (Adaptive Ensemble Guard for Injection Shielding). AEGIS extracts instruction-sensitive projectors to identify malicious instructions and leverages a Unified Multi-Layer Consensus mechanism that aggregates topologically distinct signals across the network depth. Empirical evaluations show that AEGIS achieves remarkable detection performance against both heuristic and optimization-based attacks compared to baselines, highlighting its potential to mitigate IPI. Code is available at https://github.com/xaddwell/AEGIS
