Search papers, labs, and topics across Lattice.
FocusMem introduces a novel latent memory architecture for GUI agents that separates content retention, readout, and trust mechanisms to enhance memory efficiency and decision-making. By employing a role-aware content basis, state-conditioned readout, and a lightweight trust gate, the method addresses key limitations of existing memory systems, such as loss of important details and the influence of irrelevant information. Evaluated across five benchmarks, FocusMem significantly outperforms traditional fixed-memory approaches, demonstrating that effective latent memory hinges on selective retention and contextual relevance.
Memory systems that intelligently filter and prioritize information can drastically improve GUI agent performance, as shown by FocusMem's superior results across multiple benchmarks.
GUI agents must remember both useful experience from earlier tasks and unfinished progress in the current interaction. Latent memory offers a compact solution by compressing multimodal trajectories into a few continuous tokens. Existing methods, however, usually map each trajectory to one fixed memory block and train it mainly through next-action supervision. This creates three practical problems: important details may be lost during compression, the same memory block must serve different decision stages, and irrelevant retrieved trajectories may still mislead the agent. We introduce FocusMem, which separates these responsibilities within a compact latent-memory interface. A role-aware content basis encourages episodic memory to retain reusable experience and working memory to retain task progress. A state-conditioned readout generates a decision-specific view of the same stored evidence, while a lightweight trust gate can suppress memory blocks that appear irrelevant to the current step. All components are trained while the GUI policy remains frozen. Across five GUI-agent benchmarks, FocusMem consistently outperforms a fully matched action-only fixed-memory baseline and prior latent memory adaptations. Further analysis shows that semantic and functional supervision preserve complementary information, state-conditioned readout is more robust as surrounding trajectory context grows, and the trust gate reduces the harm caused by injected irrelevant episodic evidence. These results show that effective latent memory depends not only on compressing past interaction, but also on what is retained, what is exposed, and what is allowed.
