Search papers, labs, and topics across Lattice.
This paper explores the generator-eraser paradox in LLM-assisted dialect resource creation, highlighting the dual role of large language models in both facilitating and threatening linguistic diversity. By integrating insights from variationist sociolinguistics and corpus linguistics, the authors propose a theoretical framework and derive 12 community guidelines aimed at mitigating risks associated with dialect erasure. A case study on Arabic dialects illustrates the practical application of these guidelines, addressing challenges such as diglossia and orthographic variability while promoting community governance.
LLMs could either preserve or erase dialects, and this paper offers 12 guidelines to ensure they empower rather than homogenize linguistic diversity.
Dialect resources occupy a unique position at the intersection of scientific description, cultural preservation, and computational infrastructure. Large language models offer powerful capabilities for accelerating dialect resource development through retrieval-grounded drafting, corpus navigation, metadata enrichment, and annotation workflow support. However, the same systems pose substantial risks: they can contribute to dialect erasure by privileging prestige varieties, homogenizing orthography, and enabling synthetic feedback loops that reduce linguistic diversity over time. These risks are particularly acute for language varieties characterized by diglossia, limited written standardization, or marginalized speaker communities. This paper makes three contributions. First, we integrate insights from variationist sociolinguistics and corpus linguistics to formalize the generator-eraser paradox as a theoretical framework for understanding the dual nature of LLM-assisted dialect work. Second, we derive 12 community guidelines that operationalize this framework into implementable design requirements for dialect resource creation and documentation. Third, we provide an in-depth case study of Arabic dialects, including a structured comparison of widely used resources, to demonstrate how these guidelines address language-specific challenges including diglossia, orthographic variability, and community governance. The contribution is conceptual and operational rather than experimental, with the goal of enabling dialect communities and resource builders across languages to adopt LLMs without sacrificing authenticity, variation, or sovereignty.
