Search papers, labs, and topics across Lattice.
This paper investigates the "deliteralization hypothesis" in machine translation, which posits that translations become less literal through drafting and revision. They compared the literality of human translations, NMT systems, and LLMs across 54 language pairs using a Synthetic Literality Index. The key finding is that while human translations are still less literal, LLMs deliteralize monotonically when iteratively revising their own output, but surprisingly, LLMs also tend to re-literalize idiomatic human formulations when post-editing.
LLMs, unlike humans, ironically re-literalize idiomatic expressions when post-editing human translations, revealing a surprising divergence in their understanding of translation quality.
The recent shift from dedicated NMT systems to general-purpose LLMs has reshaped machine translation, with LLMs reported to produce more fluent, less literal output than their predecessors. We test whether this shift extends to the deliteralization hypothesis, the long-standing claim from translation studies that translations become progressively less literal as they are drafted and revised. Using the WMT24++ dataset, we compare the literality of human translations and post-editions to that of two NMT systems and six LLMs across 54 language pairs and three tasks: direct translation, iterative self-revision, and post-editing of human drafts. Literality is measured via a validated Synthetic Literality Index built from six heuristics. We find that (i) human translations remain significantly less literal than those of all tested MT systems, though recent LLMs narrow the gap; (ii) when prompted to iteratively revise their own output, LLMs deliteralize monotonically, providing the first evidence that the hypothesis applies natively to LLM generation; and (iii) as post-editors, LLMs invert the revision triggers of human post-editors, tolerating literal drafts and targeting idiomatic human formulations for revision.
