Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces DHPLT, a large-scale multilingual diachronic corpus comprising web-crawled data in 41 languages across three time periods (2011-2015, 2020-2021, 2024-present). This resource addresses the scarcity of multilingual diachronic corpora for semantic change modeling, particularly for languages beyond the high-resource ones. The authors provide pre-computed word embeddings and lexical substitutions, facilitating research into semantic change across diverse languages and timeframes.
Unlock semantic change research in 41 diverse languages with DHPLT, a new open diachronic corpus filling the gap beyond the usual high-resource suspects.
In this resource paper, we present DHPLT, an open collection of diachronic corpora in 41 diverse languages. DHPLT is based on the web-crawled HPLT datasets; we use web crawl timestamps as the approximate signal of document creation time. The collection covers three time periods: 2011-2015, 2020-2021 and 2024-present (1 million documents per time period for each language). We additionally provide pre-computed word type and token embeddings and lexical substitutions for our chosen target words, while at the same time leaving it open for the other researchers to come up with their own target words using the same datasets. DHPLT aims at filling in the current lack of multilingual diachronic corpora for semantic change modelling (beyond a dozen of high-resource languages). It opens the way for a variety of new experimental setups in this field. All the resources described in this paper are available at https://data.hplt-project.org/three/diachronic/, sorted by language.