Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces InSight, a benchmark designed for agentic claim verification in interactive visualizations, addressing the limitations of existing static benchmarks. By utilizing a dataset of 21,349 claims from human-authored narratives in dynamic web environments, the authors enable models to assess the validity of claims based on user-driven interactions with visual data. The evaluation of state-of-the-art models highlights that the task of interactive verification poses significant challenges, underscoring the need for advanced reasoning capabilities in AI systems.
Interactive verification of claims in visual data is far more complex than traditional static evaluations reveal, with state-of-the-art models struggling to meet the challenge.
Vision Language Models have demonstrated remarkable proficiency in interpreting static visual artifacts, but modern data analysis is inherently dynamic, requiring the active interrogation of interactive environments. Existing benchmarks are predominantly constrained to static imagery and one-shot question answering and fail to capture the epistemic demands of this domain, where evidence is frequently occluded, distributed across linked views, or conditionally revealed through user agency. In this paper, we introduce InSight, a benchmark for agentic claim verification over interactive visualizations. The dataset consists of 21,349 claims derived from human-authored analytical narratives and grounded in fully interactive web-based environments. Agents must navigate these environments to determine whether a natural language claim is supported, refuted or not verifiable given the available evidence. Unlike traditional evaluations, InSight treats interaction traces as intrinsic proxies for reasoning, enabling a rigorous audit of how models seek and synthesize visual evidence. We evaluate state-of-the-art models, revealing that interactive verification remains a non-trivial challenge. We release InSight at https://github.com/maevehutch/insight.