Search papers, labs, and topics across Lattice.
This paper introduces Ancient-Bench, a novel benchmark for ancient Chinese artifact text recognition that addresses the limitations of existing benchmarks by providing a comprehensive dataset of 2,700 images across three dimensions: multi-millennial, multi-medium, and multi-script. The authors establish three tailored annotation standards to ensure fair evaluation of models dealing with the complexities of ancient texts. Experimental results show that current Vision-Language Models and OCR-specialist models struggle with recognizing ancient texts, highlighting the need for further advancements in this area.
Ancient-Bench reveals that even advanced models fail to solve the challenges of recognizing ancient Chinese texts, underscoring a critical gap in AI capabilities.
Ancient Chinese artifact text recognition is fundamental to heritage digitization, and benchmarks for ancient texts are essential for evaluating current model capabilities. However, existing benchmarks suffer from''fragmentation'', manifested in limited temporal coverage, limited medium diversity, and incomplete script types. Therefore, we present Ancient-Bench, a comprehensive benchmark of 2,700 images for ancient Chinese artifact text recognition, featuring three dimensions: Multi-millennial (spanning 3,000 years of character evolution), Multi-medium (covering nine artifact categories), and Multi-script (encompassing seven historical script forms). To enable consistent and fair evaluation across heterogeneous media, we further define three annotation standards tailored to the medium-specific characteristics of ancient texts: symbol standardization, character standardization, and parsing standardization. Extensive experiments on Ancient-Bench covering general Vision-Language Models (VLMs) and OCR-specialist models reveal that ancient Chinese artifact text recognition remains fundamentally unsolved, with persistent challenges in variant characters, specialized symbols, and hallucination. The dataset is available at https://github.com/SCUT-DLVCLab/Ancient_Bench.
