Search papers, labs, and topics across Lattice.
This paper introduces AgentGUI, a locally hosted graphical user interface designed to enhance human oversight of long-running AI agents by providing rich visualizations and effective steering capabilities. The user study revealed that AgentGUI significantly reduces the time required to identify key elements from agent traces by 38%, while its automated drift prevention feature improved task completion rates by up to 34 percentage points across various model sizes. By integrating with both open-source and frontier agent frameworks, AgentGUI addresses the critical gap in human-centered interfacing as AI capabilities expand.
AgentGUI cuts the time to analyze AI agent performance by 38% and boosts task completion rates by up to 34 percentage points, making human oversight of AI agents more efficient than ever.
AI agents are increasingly adept at tackling complex, long-running tasks. With the rapid surge of autonomous capabilities, human oversight is systematically lagging behind due to limited human-centered interfacing. Aiming to address this, we introduce AgentGUI, a user-friendly, locally hosted GUI for seamlessly observing and steering AI agents amid multiple concurrent, long-running sessions. AgentGUI features 1) rich agent trajectory visualizations, 2) effective manual and automated steering, and 3) integration with and coordination between open-source and frontier agent frameworks. A controlled user study demonstrates statistically significant reduction in the time it takes to identify key elements from agent traces (38% faster, p = 0.023). In a preliminary experiment, AgentGUI's automated drift prevention feature raises the task completion rate of small local agents by as high as 34pp across a 0.8B--9B model ladder (N=50 runs per model). AgentGUI is publicly available through its project website (https://agent-gui-project.github.io) and open-source repository (https://github.com/eth-medical-ai-lab/agent-gui), along with a demo video (https://youtube.com/watch?v=GSDyxN1gTF0).
