Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To systematically evaluate multimodal prompt injection against tool-using agents, the authors built MMPIBench to track adversarial instructions through perception, planning, and execution across six frameworks and five foundation models. While end-to-end visual attack completion is rare (~1%), models attempt execution in 12.8% of runs, revealing that safety margins rely almost entirely on model-level planning refusals rather than framework-level isolation or perception failures. Crucially, the same safeguards largely vanish on raw audio inputs, where attack completion rates skyrocket to 49% across supported systems and up to 75% for a single frontier model.
Agentic prompt injections fail 99% of the time in vision due to model-level planning defenses, yet the exact same attacks succeed up to 75% of the time when routed through audio channels.
Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services. Most of these agents also read images, which gives an attacker a way to put text into the agent's context without going through the user. We present MMPIBench, a reproducible benchmark that measures what happens next. It delivers a fixed set of attacks through six visual carriers (OCR text, overlays, EXIF metadata, QR codes, fake interfaces, and hybrids) and records how far each injected instruction travels through the agent, from perception through planning to the tool call. Across 720 runs covering six frameworks, five foundation models, six carriers, and four attacker objectives, attacks complete in approximately 1% of runs but are attempted in 12.8%, and the gap is closed almost entirely at the planning step, where the model reads the injected instruction and declines to act on it. The model matters far more than the framework for whether an instruction is acted on. One model never attempts an attack and recognizes the injection in 59.7% of runs, while two others attempt in 23.6%. We then extend the benchmark to audio, the only other raw perceptual channel current frontier models accept. Only two of the five models ingest audio and only three of the six frameworks deliver it, but where the signal arrives the attack completes in 49% of cells, and in 75% for one model. Reporting completion alone therefore understates exposure, and perceptual channels beyond vision are narrower but much less defended.