Search papers, labs, and topics across Lattice.
This paper introduces MV-Bench, a benchmark specifically designed to evaluate multimodal large language models (MLLMs) on their ability to construct coordinated multi-view interfaces, a task that requires complex reasoning about data semantics and interaction logic. By utilizing Tableau workbook files as a ground truth, the authors systematically assess five state-of-the-art MLLMs, revealing that while these models can reproduce visual layouts with 75.45% accuracy, they struggle significantly with data binding (21.71%) and interaction completeness (11.68%). The findings highlight a critical gap in MLLM capabilities, emphasizing the need for improved methods to generate the underlying data semantics and interactive logic essential for effective multi-view visualizations.
Current MLLMs excel at visual reproduction but falter in generating the necessary data semantics and interaction logic for coordinated multi-view interfaces.
Multimodal large language models (MLLMs) are increasingly expected to automate visualization development by generating code directly from visual designs. However, existing evaluations mainly focus on single-chart generation and overlook coordinated multi-view interface construction, which requires joint reasoning about data semantics, view coordination, and interaction logic. Consequently, MLLM capabilities in this setting remain underexplored, and the field lacks a dedicated benchmark for systematic assessment. We introduce MV-Bench, a benchmark for evaluating MLLMs on coordinated multi-view interface construction. Instead of relying on incomplete or inconsistent open-source implementations, we use Tableau workbook files as ground truth because they explicitly encode data bindings, visual mappings, and interactions. We develop a multi-stage pipeline that converts these specifications into executable web interfaces through structured intermediate representations. The benchmark contains 92 base interfaces and 1,048 verified instances created by recombining chart types, datasets, and interaction patterns. Each instance includes executable code, a rendered interface, a dataset, and interaction annotations. We evaluate five state-of-the-art MLLMs in a single-pass setting using metrics for visual fidelity, data binding correctness, and interaction completeness. The strongest model achieves 75.45 percent accuracy in visual layout reproduction, but only 21.71 percent in data binding and 11.68 percent in interaction completeness. These results show that current MLLMs can reproduce visual appearance but remain limited in generating the data semantics and interactive logic required by coordinated multi-view interfaces. Iterative refinement improves code executability but does not substantially reduce the gap in data binding and interaction generation.
