Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the vulnerability of tool-using LLM agents to indirect prompt injection attacks, revealing a significant "framing gap" where the same information can be exfiltrated depending on its presentation. Through experiments with six models, the authors demonstrate that while overt injection attempts are consistently rejected, reframing these attempts as legitimate requests can lead to successful exfiltration, with GPT-4o's success rate jumping from 0% to 100%. The findings highlight that effective defenses must focus on constraining the destination or isolating capabilities rather than relying solely on the model's ability to recognize attacks.
Indirect prompt injections can exploit framing strategies to bypass defenses, turning a 0% success rate into 100% with simple rewording techniques.
A tool-using LLM agent that reads attacker-controlled web content while holding a secret faces indirect prompt injection: the content may make it exfiltrate the secret. In a safe synthetic lab (canary secret, mock tools, matched clean-vs-poisoned metric) we report the framing gap: across six models, ten overt injection classes are refused (gpt-4o 0%), but reframing the identical leak as a mandatory integrity signature, config field, or look-alike"trusted"host drives gpt-4o 0% to 100%. The attack is cheap, and its cost is three-level: paraphrasing a known mechanism is trivial (96% at 3 wordings), swapping the field inside a known-effective template is also cheap (up to 60%), while authoring a fresh page around a new mechanism is hard (0/130) -- the reusable asset is the template, not the mechanism. An ablation shows the mechanism is instruction/data confusion, not defeated alignment: removing the confidentiality policy leaves base attacks at 0% and moves reframing only 31.9% to 38.1%. What closes the gap is payload-blind checks: a destination allow-list (0%, when destinations are closed) and a capability-isolating planner/reader split (0%). A broad"in any form"policy clause also closes it at the acting model (to 0%) but is brittle (dropping the catch-all reopens it to 48.8%). A published fine-tuning defense (SecAlign, CCS 2025) does not close it on a tool agent (32.5%, positive-control-validated), nor does channel separation (38.8%); an output-normalizing guard loses to a held-out encoding (ROT13, 100%). Robustness comes from constraining the destination or isolating the capability, not from the acting model recognizing the attack.