Search papers, labs, and topics across Lattice.
The authors introduce SumTablets, a large dataset of 91,606 Sumerian cuneiform tablets represented as Unicode glyph-transliteration pairs, along with period and genre metadata. This dataset addresses the lack of a comprehensive resource for applying NLP methods to Sumerian transliteration, a crucial but time-consuming task in Assyriology. By fine-tuning an autoregressive language model on SumTablets, they achieve a 97.55 character-level F-score, showcasing the dataset's utility for deep learning-based transliteration.
A new dataset of nearly 100,000 Sumerian tablets paired with transliterations unlocks the potential for NLP to accelerate understanding of this ancient civilization.
Transliterating Sumerian is a key step in understanding Sumerian texts, but remains a difficult and time-consuming task. With more than 100,000 known texts and comparatively few specialists, manually maintaining up-to-date transliterations for the entire corpus is impractical. While many transliterations have been published online thanks to the dedicated effort of previous projects, the lack of a comprehensive, easily accessible dataset that pairs digital representations of source glyphs with their transliterations has hindered the application of natural language processing (NLP) methods to this task.To address this gap, we present SumTablets, the largest collection of Sumerian cuneiform tablets structured as Unicode glyph–transliteration pairs. Our dataset comprises 91,606 tablets (totaling 6,970,407 glyphs) with associated period and genre metadata. We release SumTablets as a Hugging Face Dataset.To construct SumTablets, we first preprocess and standardize publicly available transliterations. We then map them back to a Unicode representation of their source glyphs, retaining parallel structural information (e.g., surfaces, newlines, broken segments) through the use of special tokens.We leverage SumTablets to implement and evaluate two transliteration approaches: 1) weighted sampling from a glyph’s possible readings, 2) fine-tuning an autoregressive language model. Our fine-tuned language model achieves an average transliteration character-level F-score (chrF) of 97.55, demonstrating the potential use of deep learning methods in Assyriological research.
