Search papers, labs, and topics across Lattice.
This paper introduces ActReal, a framework that enables system-level mobile agents to execute physical-action attacks by converting semantic actions into valid touch and IMU events. The significance lies in its ability to circumvent existing automation detection methods that rely on analyzing touch trajectories and action timing, demonstrating that a privileged agent can effectively generate synchronized touch and sensor signals. ActReal achieves a remarkable mean attack success rate of 77.5%, and maintains a 71.1% success rate even against detectors monitoring both touch and IMU data.
ActReal reveals that system-level mobile agents can successfully evade automation detection by generating synchronized touch and IMU signals, achieving over 77% attack success rates.
System-level mobile agents are evolving from fixed scripts into adaptive systems that continuously observe interfaces, reason, and adjust their actions, allowing automated attacks to navigate dynamic UIs and complete complex tasks. Existing applications detect automation using touch trajectories, action timing, and the physical coupling between touch and inertial measurement unit (IMU) signals. However, a privileged system-level agent executor can control both touchscreen input and application-visible sensor delivery, enabling it to jointly generate time-aligned touch and six-axis IMU signals and evade these defenses. We present ActReal, a physical-action attack framework for system-level mobile agents. ActReal converts semantic agent actions into task-valid touch and IMU events using genuine-trajectory adaptation and physics-guided IMU generation. ActReal achieves a mean event-level attack success rate of 77.5\%; even when detectors jointly observe touch and IMU, its attack success rate remains 71.1\%.
