Search papers, labs, and topics across Lattice.
This paper introduces TABVERSE, a controlled benchmark designed to evaluate the effects of table representation on the performance of Large Language Models (LLMs) and Vision-Language Models (VLMs) across various structural formats. By isolating the content from format, layout, and modality, the authors systematically assess model performance on tasks such as Question Answering, Structural Understanding Capability, and Structure Reconstruction. The results reveal that the choice of representation significantly influences understanding, with structured text formats generally yielding better performance than rendered images, highlighting the importance of representation in table evaluation.
Representation choice can drastically alter model performance in table understanding tasks, revealing that structured text often outperforms rendered images.
Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly evaluated on table reasoning tasks, but the role of table representation remains under-explored. In practice, the same table content may appear in different structural formats, such as HTML, Markdown, and LaTeX, or as rendered images. However, existing evaluations often let content, format, layout, and modality vary together, making it difficult to isolate representation effects. We introduce TABVERSE, a controlled multimodal table benchmark that aligns the same table content across multiple structural formats and rendered images, with question category and difficulty tags. This design enables systematic evaluation of representation effects while holding table content fixed. We evaluate LLMs and VLMs across three tasks: Question Answering (QA), Structural Understanding Capability (SUC), and Structure Reconstruction (SR). Our results show that representation choice substantially affects table understanding. Models generally perform better with structured text than with rendered images, but the size of this gap depends on the task, model, and format. HTML is often the most robust text format, while row-sensitive structural tasks and syntactically usable LaTeX reconstruction remain challenging. These findings show that table representation is a key factor in reliable table evaluation.
