Search papers, labs, and topics across Lattice.
This paper introduces ArtiFact, a large-scale multi-modal dataset comprising over 650,000 museum records that integrates tables, text, and images from prominent cultural institutions. The dataset addresses a critical gap in the database community by providing a resource for exploring cross-modal error detection and semantic query processing, highlighting the challenges in accurately detecting domain-specific errors and handling complex queries. Key findings reveal that existing systems struggle with nuanced cultural queries and error detection, underscoring the need for improved methodologies in multi-modal data management.
Current systems falter in detecting subtle errors and processing complex queries in cultural heritage data, revealing significant gaps in multi-modal data management capabilities.
Multi-modal data management has emerged as a central research topic in the database community, spanning data integration, semantic query processing, and data quality assessment. Despite this growing interest, the community lacks large-scale, real-world datasets combining tables, text, and images. We present ArtiFact, a multi-modal cultural heritage dataset of 651045 museum records collected from the Metropolitan Museum of Art, the Art Institute of Chicago, and the Rijksmuseum. We demonstrate the utility of ArtiFact through two downstream tasks. For cross-modal error detection, we introduce a curated taxonomy of seven error categories injected into 130209 records and show that reliably detecting subtle domain-specific errors such as material anachronisms and temporal shifts remain an open challenge. For semantic query processing, we show that current systems struggle with queries involving cultural proximity, ambiguous object types, and historically contingent terminology. Our results position ArtiFact as a challenging benchmark for multi-modal data management research.
