Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a pipeline that utilizes Vision-Language Models (VLMs) to automatically extract structured metadata from historical auction catalogs, addressing the challenge of variable internal formatting in these documents. By evaluating different prompt strategies and decoding frameworks on a manually annotated test set, the authors benchmark the performance of various deployment modes, revealing that commercial endpoints provide the highest accuracy while institutional gateways offer a privacy-preserving alternative. The findings indicate that while some human intervention is still needed, VLMs can effectively facilitate large-scale analysis of auction catalogs, significantly enhancing provenance research in the art market.
Unlocking historical auction catalogs with VLMs could revolutionize provenance research by automating the extraction of structured metadata from variable formats.
For provenance research and art market studies, auction catalogs are an essential resource to trace specific objects over time and space. While historical auction catalogs follow established domain conventions, their internal formatting remains highly variable, and their large-scale analysis is currently restricted by the lack of machine-readable representations of the auction lots. We propose a pipeline to automatically extract structured lot-level metadata from German Sales, a large database of historical auction and sales catalogs from the 19th and 20th centuries. Using a manually annotated test set of representative catalog pages, we evaluate Vision-Language Models (VLMs) under varying prompt strategies and constrained decoding frameworks. To reflect the practical constraints faced by cultural heritage institutions, including budget, compute resources, and data privacy requirements, we benchmark the methods across different deployment modes ranging from commercial providers to locally hosted, quantized models. We find that commercial endpoints establish the performance ceiling, while institutional gateways offer a viable, privacy-preserving alternative. Local deployments remain feasible, but strictly require enforcing the output structure during generation to guarantee a valid JSON format. While varying degrees of human-in-the-loop correction are still necessary, this work demonstrates that a VLM-based pipeline can successfully unlock historical auction catalogs for large-scale automated analysis.