Search papers, labs, and topics across Lattice.
The paper introduces SAYRE, a scene-aware document synthesis framework designed to enhance Key Information Extraction (KIE) by generating scalable training data without the need for hand-crafted templates. By leveraging exemplar documents, SAYRE captures specific content patterns and layout conventions to create document-schema-annotation triples, while also incorporating error-driven generation to address real-world failure cases. Experimental results demonstrate that SAYRE significantly improves the performance of on-device Large Multimodal Models (LMMs), particularly in constrained and open-category KIE tasks, showcasing its effectiveness in reducing field-level errors.
SAYRE's innovative approach to synthesizing KIE training data leads to substantial performance gains for on-device models, particularly in challenging extraction scenarios.
Key Information Extraction (KIE) converts visually rich documents into structured data, but practical deployment remains challenging: strong performance often relies on costly on-server Large Multimodal Models (LMMs), while compact locally deployable models lack sufficient KIE supervision. We present SAYRE, a scene-aware document synthesis framework for generating scalable KIE training data without hand-crafted template design. Given a few exemplar documents, SAYRE captures category-specific content patterns and layout conventions to synthesize document-schema-annotation triples. It further introduces error-driven generation, which expands real-world failure cases into hard training examples while preserving their structural patterns. Experiments on constrained- and open-category KIE show that SAYRE consistently improves Qwen3-VL backbones and achieves the strongest overall performance among on-device LMMs. Data scaling experiments show an overall upward trend as more synthesized data is introduced, especially for smaller models and open-category extraction. Error analysis further shows that synthesized training reduces field-level errors by improving schema-aware extraction over dense tables, business identifiers, and contract clauses. These results establish scene-aware synthesis as an effective data-centric approach for improving practical multimodal KIE.
DAMO