Search papers, labs, and topics across Lattice.
This paper investigates the gap between understanding and generating dialectal English in large language models by introducing DiaLLM, which employs continual pretraining on the International Corpus of English alongside various post-training paradigms and alignment strategies. The findings reveal a dissociation between dialectal robustness and generation, with explicit variety-targeted adaptation yielding outputs that are recognized as dialectal but not necessarily preferred by human evaluators. The study highlights the complexity of aligning model outputs with dialectal expectations, emphasizing the need for improved reward designs and resources to effectively bridge this gap.
Dialectal generation in LLMs is not just about understanding; explicit adaptation methods can produce recognizable dialects, yet they may not align with human preferences.
Large language models increasingly \emph{understand} dialectal English, yet still \emph{produce} only standard, US-leaning English, leaving dialectal generation, the harder half of the problem, largely unaddressed. We introduce \textbf{DiaLLM}, which continually pretrains three open-weight language model families on the International Corpus of English and applies implicit and explicit post-training paradigms, each combined with three model alignment strategies, giving the first controlled comparison of these components across Australian, Indian, and Northern British English. Our results reveal that dialectal robustness and generation are \emph{dissociated}: benchmarks are shaped by continual pretraining and SFT, while alignment visibly reshapes generation in ways benchmarks do not capture. Explicit variety-targeted adaptation produces output reliably recognised as dialectal and preferred over broad alignment, yet the method that most aggressively optimises the dialectal reward is not preferred by human evaluators. Independent linguistic analysis corroborates this reward-quality gap, most clearly on two of the three families. No single alignment method dominates, and closing the gap will require richer reward designs and continued investment in dialectal resources. We release all code, checkpoints, and preference datasets.
