Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a trace-level evaluation framework to compare human and GUI-agent behavior across task outcome/effort, query formulation, and interface navigation. Applying this framework to a production audio-streaming search application, the authors found that while a state-of-the-art GUI agent achieved comparable task success and query alignment to human participants, it exhibited significantly different navigation strategies. Specifically, humans demonstrated content-centric, exploratory behavior, whereas the agent was more search-centric and followed lower-branching paths.
GUI agents can achieve the same task success as humans in production search systems, but their interaction patterns are fundamentally different, raising concerns about their validity as user proxies.
LLM-driven GUI agents are increasingly used in production systems to automate workflows and simulate users for evaluation and optimization. Yet most GUI-agent evaluations emphasize task success and provide limited evidence on whether agents interact in human-like ways. We present a trace-level evaluation framework that compares human and agent behavior across (i) task outcome and effort, (ii) query formulation, and (iii) navigation across interface states. We instantiate the framework in a controlled study in a production audio-streaming search application, where 39 participants and a state-of-the-art GUI agent perform ten multi-hop search tasks. The agent achieves task success comparable to participants and generates broadly aligned queries, but follows systematically different navigation strategies: participants exhibit content-centric, exploratory behavior, while the agent is more search-centric and low-branching. These results show that outcome and query alignment do not imply behavioral alignment, motivating trace-level diagnostics when deploying GUI agents as proxies for users in production search systems.