Search papers, labs, and topics across Lattice.
This paper introduces State-aware Visualization-of-Thought (SVoT), a reinforcement learning framework designed to enhance spatial reasoning in Multimodal Large Language Models (MLLMs) by generating verifiable intermediate states and visualizations. By integrating transition reasoning chains and employing Group Relative Policy Optimization (GRPO) with fine-grained reward structures, SVoT enables the model to effectively verify action preconditions and effects during reasoning tasks. The framework demonstrates significant improvements in multi-hop spatial reasoning, achieving up to a 65% absolute accuracy gain on out-of-distribution test sets across newly established domains, including Pacman and Gather.
SVoT achieves a remarkable 65% accuracy boost in spatial reasoning tasks by making intermediate states verifiable and integrating transition reasoning.
Spatial reasoning remains a challenge for Multimodal Large Language Models (MLLMs), as it requires reliable multi-hop inference over both intermediate states and state transitions. Current studies often leave intermediate states unverified and treat state transitions as implicit processes, which limits reliability in multi-hop spatial reasoning. To address this, we propose State-aware Visualization-of-Thought (SVoT), a reinforcement learning framework that generates interleaved, verifiable intermediate states and visualizations. SVoT integrates transition reasoning chains into the generation processes, enabling the model to verify action preconditions and effects through interleaved textual and visual reasoning. We train SVoT via Group Relative Policy Optimization (GRPO), instantiating verification through reward design and evaluating the efficacy of different fine-grained rewards. As existing benchmarks reduce state transitions to single-variable updates, substantially simplifying the problems, we establish five domains by extending classical environments and introducing two novel domains, Pacman and Gather, that require multi-object interactions and numerical reasoning. These domains support systematic evaluation of multi-hop spatial reasoning with quantitative verification of generated intermediate states and transition reasoning. SVoT with transition-aware supervision achieves state-of-the-art performance across the introduced domains, yielding up to a 65% absolute accuracy gain on out-of-distribution test sets.
