Search papers, labs, and topics across Lattice.
This paper introduces TriViewBench, a novel benchmark designed to assess the scalability of Multimodal Large Language Models (MLLMs) in visual reasoning tasks with controlled structural complexity. Through the evaluation of 18 MLLMs across various complexity levels and reasoning categories, the study uncovers a consistent capability hierarchy and significant performance degradation as task complexity increases, particularly in Object Counting and Global Recovery tasks. The findings highlight critical limitations in current MLLMs' ability to handle multi-view reasoning, suggesting that issues stem from spatial representation rather than reasoning strategies.
MLLMs face severe scalability limitations, with performance dropping by up to 80% on complex visual reasoning tasks, revealing a critical gap in their structural reasoning capabilities.
Multimodal Large Language Models (MLLMs) demonstrate strong performance on standard visual question answering benchmarks, yet their scalability under controlled structural complexity remains poorly understood. We introduce TriViewBench, a controlled three-view visual reasoning benchmark constructed from synthetic 3D scenes with explicitly parameterized object count and occlusion. The benchmark contains 1,923 scenes and over 14K Question-Answer (QA) pairs organized into four complexity levels and three reasoning categories: Local Decision, Object Counting, and Global Recovery. We evaluate 18 open- and closed-source MLLMs under a unified prompting protocol. All 18 models exhibit an identical capability hierarchy without exception (Local Decision > Object Counting > Global Recovery), and performance degrades monotonically with complexity: Local Decision tasks decline modestly (12.11% relative drop), while Object Counting degrades substantially (59.14%) and Global Recovery collapses severely (80.02%). Error analysis on Object Counting reveals two mechanistically independent failure modes: single-view tasks are dominated by undercounting due to occlusion blindness, whereas the multi-view task reverses to overcounting due to cross-view identity confusion. Chain-of-Thought (CoT) prompting yields near-zero overall benefit ($螖= -0.16\%$) and its effect on Global Recovery is strongly capability-gated, suggesting that the bottleneck lies in cross-view spatial representation rather than reasoning strategy. These findings reveal fundamental scalability limitations in current MLLMs and position TriViewBench as a controlled diagnostic framework for analyzing structural reasoning failures.
