Search papers, labs, and topics across Lattice.
This paper introduces the Visual Spatial-Temporal Reasoning Benchmark (ViSTR-Bench), a novel evaluation suite designed to assess the qualitative reasoning abilities of Multimodal Large Language Models (MLLMs) using continuous visual cues in dynamic scenes. The benchmark includes 15 distinct subtasks and 1,340 video question-answer pairs, focusing on Motion Perception, Spatial Relations, Outcome Prediction, and Physical Dynamics. Evaluation results indicate that while MLLMs demonstrate strong general video understanding, they still significantly lag behind human performance in complex spatial-temporal reasoning tasks.
Despite advances in MLLMs, they still struggle with dynamic reasoning, falling far short of human capabilities in interpreting continuous visual cues.
Multimodal Large Language Models (MLLMs) have achieved remarkable success across diverse expert-level tasks, but they still struggle with fundamental abilities that humans naturally develop through continuous observation of the real world, such as spatial perception and dynamic reasoning. Recent studies have recognized this gap and introduced dedicated benchmarks to evaluate the spatial-temporal capabilities of MLLMs. However, existing benchmarks mostly focus on static scenes or require exact quantitative predictions, leaving intuitive reasoning from temporal cues largely underexplored. In this paper, we introduce the Visual Spatial-Temporal Reasoning Benchmark (ViSTR-Bench), a novel evaluation suite designed to systematically assess whether MLLMs can perform qualitative reasoning from continuous visual cues in dynamic scenes. Guided by the principles of temporal emphasis, reasoning orientation, and qualitative evaluation, ViSTR-Bench establishes a comprehensive four-dimensional evaluations covering Motion Perception, Spatial Relations, Outcome Prediction, and Physical Dynamics. The benchmark comprises 15 distinct subtasks and 1,340 high-quality video question-answer pairs spanning diverse tabletop, indoor, and outdoor scenarios. Extensive evaluations of a broad spectrum of state-of-the-art proprietary, open-source, and specialized spatial MLLMs reveal that, despite their strong general video understanding capabilities, current models still face substantial bottlenecks in complex spatial-temporal reasoning and remain far below human performance.
