Search papers, labs, and topics across Lattice.
This paper introduces DataVista, the first benchmark for evaluating multimodal large language models (MLLMs) on data video understanding, a domain crucial for news and business analysis that requires interpreting animated charts and integrating evidence across time and narrative. Through systematic evaluation of 19 MLLMs on 961 real-world data videos and 6,775 questions, the study reveals that even top models like Gemini-3.1-Pro fall significantly short of human expert performance, particularly in causal reasoning and narrative structure. The analysis highlights common failure modes and demonstrates that while increasing frame counts and adding subtitles improve basic perception and temporal reasoning, they offer limited gains for higher-level narrative comprehension.
Even state-of-the-art MLLMs struggle to grasp the nuanced data interpretation and narrative coherence required for data videos, with significant performance gaps in causal reasoning and narrative structure.
Data video is a media form that integrates data visualization with video narrative, widely adopted in news reporting and business analysis. Compared with general video understanding, data video understanding places greater emphasis on accurately reading data from animated charts, integrating evidence across charts and time, and understanding how narrative organization and visual design communicate information. Yet existing benchmarks target either general videos or static charts, and data video understanding has not been systematically evaluated. We present DataVista, the first benchmark for data video understanding, containing 961 real-world data videos and 6,775 evaluation questions organized under a three-level progressive capability framework (data perception, temporal reasoning, narrative understanding) with 10 fine-grained question types across five topic domains. Systematic evaluation of 19 mainstream MLLMs shows that the best-performing model, Gemini-3.1-Pro, achieves 70.0% overall accuracy, still far below human expert performance, with models performing worst on Causal Reasoning and Narrative Structure. Increasing frame counts and adding subtitles mainly benefit data perception and temporal reasoning, with limited gains in narrative understanding. Further analysis of model responses identifies typical failure modes in chart reading, evidence judgment, and instruction understanding. The benchmark is available at https://github.com/HKUSTDial/DataVista.
