Search papers, labs, and topics across Lattice.
ReceiptBench, a new large-scale benchmark, is introduced to evaluate MLLMs on real-world receipt understanding across four hierarchical sub-tasks: basic perception, format normalization, semantic reasoning, and structure parsing. To improve performance on this benchmark, a two-stage training framework is proposed using Metric-Aware Group Relative Policy Optimization (GRPO) to enforce structural consistency. Experiments show that the GRPO-enhanced models achieve state-of-the-art results, outperforming proprietary models on complex reasoning tasks within receipt understanding.
MLLMs still struggle with real-world document understanding, but a new benchmark and reinforcement learning approach can significantly improve their ability to extract structured information from receipts.
Extracting structured information from visual documents (Visual Information Extraction, VIE) is a cornerstone of business automation. While recent Multimodal Large Language Models (MLLMs) have shown promising capabilities, existing benchmarks suffer from critical limitations in scale and realism, lack semantic granularity, and fail to cover diverse document types. To bridge this gap, we introduce ReceiptBench, a large-scale, human-annotated benchmark consisting of 10k diverse receipts, organizing information extraction into four hierarchical sub-tasks: (1) Basic Perception for raw text spotting, (2) Format Normalization for strictly following standardization instructions, (3) Semantic Reasoning for inferring implicit attributes from context, and (4) Structure Parsing for handling nested line items. Furthermore, we propose a two-stage training framework incorporating Metric-Aware Group Relative Policy Optimization (GRPO), which translates rigorous evaluation constraints into reinforcement learning signals to enhance structural consistency. Extensive experiments demonstrate that our method yields state-of-the-art performance, surpassing leading proprietary models on complex reasoning tasks. We release our datasets and code at https://github.com/wwwT0ri/ReceiptBench.
