Search papers, labs, and topics across Lattice.
The paper introduces SpatiaLQA, a new benchmark designed to evaluate the spatial logical reasoning capabilities of Vision-Language Models (VLMs) in complex, real-world indoor scenes. The benchmark comprises 9,605 question-answer pairs derived from 241 real-world indoor scenes, requiring VLMs to understand spatial relationships and logical dependencies in multi-step tasks. Experiments on 41 mainstream VLMs reveal significant shortcomings in spatial logical reasoning, which the authors address by proposing a recursive scene graph assisted reasoning method that outperforms existing approaches.
VLMs still can't reason about spatial logic in real-world scenes, but a new benchmark and scene graph method shows how to make progress.
Vision-Language Models (VLMs) have been increasingly applied in real-world scenarios due to their outstanding understanding and reasoning capabilities. Although VLMs have already demonstrated impressive capabilities in common visual question answering and logical reasoning, they still lack the ability to make reasonable decisions in complex real-world environments. We define this ability as spatial logical reasoning, which not only requires understanding the spatial relationships among objects in complex scenes, but also the logical dependencies between steps in multi-step tasks. To bridge this gap, we introduce Spatial Logical Question Answering (SpatiaLQA), a benchmark designed to evaluate the spatial logical reasoning capabilities of VLMs. SpatiaLQA consists of 9,605 question answer pairs derived from 241 real-world indoor scenes. We conduct extensive experiments on 41 mainstream VLMs, and the results show that even the most advanced models still struggle with spatial logical reasoning. To address this issue, we propose a method called recursive scene graph assisted reasoning, which leverages visual foundation models to progressively decompose complex scenes into task-relevant scene graphs, thereby enhancing the spatial logical reasoning ability of VLMs, outperforming all previous methods. Code and dataset are available at https://github.com/xieyc99/SpatiaLQA.
