Search papers, labs, and topics across Lattice.
This paper addresses the underperformance of Large Reasoning Models (LRMs) in spatial reasoning tasks by proposing a self-supervised reinforcement learning framework that enhances internal reasoning through logical coherence rather than relying on supervised fine-tuning. By introducing consistency verifiers as reward functions that assess geometric and semantic consistency, the authors demonstrate that LRMs can significantly improve their spatial reasoning capabilities without the need for labeled data. The proposed optimal transport-based RL strategy, OT-GRPO, achieves accuracy comparable to supervised methods while maintaining strong generalization across various tasks and domains.
Label-free consistency training can elevate spatial reasoning in LRMs to levels comparable with traditional supervised methods, revolutionizing how we approach model training.
Current Large Reasoning Models (LRMs) exhibit remarkable general capabilities but significantly underperform in spatial reasoning tasks. Existing approaches treat this gap as a knowledge deficit, relying on supervised fine-tuning (SFT) to ingest labeled spatial data from external vision sources or synthetic engines. In contrast, we argue that for many tasks, spatial reasoning capabilities are already present in pre-trained LRMs but require alignment through logical coherence under geometric 2D and 3D constraints. In this work, we propose a self-supervised reinforcement learning (RL) framework that targets the internal reasoning process without requiring ground-truth annotations. By formalizing the notion of consistency verifiers -- reward functions that check for geometric and semantic consistency under transformations -- we demonstrate that models can improve their spatial reasoning abilities. We use both image transformations, like flipping, and textual transformations, like swapping the order of objects in the question, and propose a new optimal transport-based RL strategy, OT-GRPO, which is a minimal-matching variant of group relative policy optimization tailored to pairwise verifiers. We show that this label-free consistency training approaches the accuracy of models trained with ground-truth supervision and achieves similar generalization across diverse tasks and data domains.
Stanford HAI