Search papers, labs, and topics across Lattice.
This paper introduces Unified Agent, a novel AI agent designed to manage interactions across multiple devices by maintaining a compact, stateful representation of user engagement and requests over time. The authors highlight the limitations of existing systems, which either lack effective state management or fail to coordinate across devices, and demonstrate that Unified Agent significantly outperforms four existing designs in various multimodal large language model (MLLM) settings. The key finding reveals that the state-design advantage of Unified Agent is robust, providing a more effective solution for cross-device, cross-time user interactions.
Unified Agent outperforms existing multi-agent systems by effectively maintaining a compact state across devices and time, revolutionizing user-agent interactions.
As capabilities rapidly increase, AI agents can move from running inside one app to acting across a user's devices over time. Yet existing agent systems still fall short in this scenario. This is because observations are scattered across devices and moments, but mainstream systems are not designed around this fact: a single agent that treats devices as tools lacks effective state management for all devices across time, and multi-agent systems coordinate across agents but do not maintain the compact carried state a cross-device, cross-time request needs. We argue that the agent should maintain an effectively designed state that organizes engagement evidence, stated facts, and the standing request in a compact, action-ready form for deciding its action given the current observation. To compare state designs, we construct a benchmark of user-agent interaction across devices and time. We instantiate this principle in Unified Agent, a stateful agent that carries interaction evidence across devices and moments and uses it with the current observation to act. In the default setting, it significantly outperforms our adaptations of four published designs. Across changes in multimodal large language model (MLLM) family, capability, and reasoning effort, it remains ahead of all compared systems, demonstrating that the state-design advantage is robust across MLLM settings. Our code and data will be publicly available on GitHub.
