Search papers, labs, and topics across Lattice.
This paper addresses the issue of unfaithful spatial reasoning in Multimodal Large Language Models (MLLMs) that can lead to inaccurate final answers due to errors in intermediate judgments. The authors introduce a modular, training-free framework that utilizes a Spatial Evidence Graph (SEG) to verify and correct reasoning chains by assessing the reliability of spatial evidence and guiding MLLMs to revise their outputs accordingly. Their approach achieves an average accuracy of 68.94% across 15 model-dataset settings, outperforming existing baselines by 8.55 percentage points, highlighting the importance of fidelity in reasoning processes for improved accuracy.
Unfaithful reasoning chains can significantly undermine the accuracy of MLLMs, but a training-free framework can effectively correct these errors and boost performance by over 8%.
Although Multimodal Large Language Models (MLLMs) have made substantial progress, their spatial reasoning may still produce intermediate judgments inconsistent with the input image, allowing errors to propagate through the reasoning chain and affect the final answer. Existing methods mainly improve spatial reasoning through training or additional spatial information, without considering whether the reasoning process itself is faithful to the model input. Our study shows that unfaithful reasoning chains significantly reduce final-answer accuracy. To address this issue, we propose a modular and training-free framework for spatial reasoning verification and correction. The framework constructs a Spatial Evidence Graph (SEG), which associates atomic spatial evidence extracted from Chain-of-Thought reasoning with visual entities, spatial relations, source steps, and visual evidence. Spatial Evidence Reliability Assessment (SERA) evaluates the reliability of visual evidence based on object existence, localization, and geometric measurements. The framework then identifies the earliest spatial evidence unit contradicted by reliable visual evidence and guides the original MLLM to revise the subsequent reasoning and final answer. Across 15 model-dataset settings, our method achieves an average accuracy of 68.94%, outperforming the compared baselines by 8.55 percentage points on average. Our code will be open-sourced.
