Search papers, labs, and topics across Lattice.
This paper investigates the impact of machine-translated data (translationese) on the training of small English language models by training models on English text translated from 24 diverse source languages. The study reveals that the source language significantly influences model behavior, with general perplexity being primarily affected by the lexical diversity of the translated corpus. Grammatical performance, however, is strongly correlated with the typological similarity between the source language and English, provided sufficient data is available.
Training on machine-translated text warps language models in predictable ways: lexical diversity drives general fluency, but grammatical accuracy hinges on source language similarity.
Machine-translated data is widely used in multilingual NLP, particularly when native text is scarce. However, translated text differs systematically from native text. This phenomenon is known as translationese, and it reflects both traces of the source language and characteristic properties of translation itself. In this paper, we study how training on machine-translated data affects small English language models, focusing on how translationese from different source languages shapes linguistic acceptability judgments and language modelling for different domains. We train models on English text translated from 24 typologically and resource-diverse source languages, enabling a systematic analysis of how source language and corpus properties influence what models learn. Our results show that the source language has a clear impact on model behavior: general perplexity is more driven by the lexical diversity of the translated corpus, while grammatical performance is strongly correlated to typological similarity to English, given enough data.
