Search papers, labs, and topics across Lattice.
This paper investigates the security vulnerabilities of third-party mobile agents powered by Vision-Language Models (VLMs), identifying two novel attack surfaces: the Screen Perception Attack Surface and the Misused Channel Attack Surface. By implementing seven distinct attacks, including subliminal text injection and screenshot tampering, the authors demonstrate that malicious apps can hijack agent actions and execute arbitrary commands without requiring elevated permissions. The findings expose a critical trust mismatch in the design of autonomous agents and underscore the necessity for enhanced security measures tailored to the unique characteristics of these systems.
Malicious apps can exploit third-party mobile agents to execute arbitrary commands without any elevated permissions, revealing a dangerous trust gap in agent design.
Third-party mobile agents powered by Vision-Language Models (VLMs) have emerged as a promising paradigm for automating smartphone interactions. These agents act as high-privilege decision-makers, perceiving device states through screenshots and executing actions via VLM reasoning, transforming how an agent app interacts with the environment (i.e., other apps or the OS). Correspondingly, this transformation introduces new attack surfaces or transforms benign/harmless interfaces into exploitable ones for mobile devices. In this paper, we summarize key differences between third-party mobile agent apps and general apps when interacting with the environment, analyze the security posture of agents, and identify two unique attack surfaces compared to general mobile apps: the Screen Perception Attack Surface, which exploits the gap between human and machine vision, and the Misused Channel Attack Surface, which intercepts or manipulates the agent's execution pipeline. We design and implement seven concrete attacks, from subliminal text injection and invisible pixel zone exploitation to screenshot tampering and host PC command injection. Our evaluation of five popular mobile agent frameworks demonstrates that a malicious app can hijack agent actions and achieve arbitrary command execution even without any privilege permissions, while remaining visually indistinguishable to users. These findings reveal a fundamental trust mismatch in autonomous agent design and highlight the urgent need for perception-aware security models on multi-tenant platforms.
