Search papers, labs, and topics across Lattice.
This paper introduces the Spatial Memory Agent (SMA), a novel framework that enhances the spatial reasoning capabilities of frozen Vision-Language Models (VLMs) without requiring parameter updates or external spatial tools. By leveraging experience-grounded runtime processes, SMA distills transferable lessons from verified spatial experiences through a method called verifier-guided reflection, assigning a Transfer Reliability Score (TRS) to each lesson. The results demonstrate that SMA significantly outperforms existing methods across multiple spatial benchmarks, establishing a new approach for improving spatial intelligence in VLMs.
SMA achieves superior spatial reasoning in frozen VLMs by converting verified experiences into transferable lessons, outperforming traditional methods without the need for parameter updates.
Spatial intelligence is becoming a foundation for embodied agents, robotic planning, and multimodal assistants. To improve the spatial reasoning ability of VLM agents, existing work has mainly followed two lines. One line uses post-training methods, such as supervised fine-tuning and reinforcement learning. Another line adopts an agentic paradigm in which the model calls external spatial tools, such as depth estimation and 3D reconstruction tools, to gather intermediate spatial evidence. We study a complementary and underexplored route: Can a frozen VLM agent improve its spatial reasoning through \textbf{parameter-update-free self-evolution}, without depending on external expert spatial tools at inference time? We present \textbf{Spatial Memory Agent (SMA)}, an \textbf{experience-grounded runtime framework} that converts verified spatial experience into reusable transferable lessons. In a verifiable spatial environment, SMA queries the frozen VLM, obtains a predicted answer and reward, and uses \textbf{verifier-guided reflection} to distill compact transferable lessons from spatial experience. SMA further assigns each lesson a \textbf{Transfer Reliability Score (TRS)}, which is initialized uniformly and calibrated from later retrieval outcomes as visit evidence of future transfer reliability. During \textbf{read-only deployment}, SMA retrieves lessons by semantic filter and similarity-TRS combined ranking, allowing the retrieved memory to guide frozen model inference. Across five representative spatial benchmarks and four base VLMs, SMA achieves the highest macro average in every base-model block and the best accuracy among the evaluated methods in most of the 20 evaluations, establishing a practical parameter-update-free path for spatial self-evolution across the evaluated frozen model scales and environments.
