Search papers, labs, and topics across Lattice.
This paper introduces a novel reinforcement learning-based post-training framework for neural machine translation (NMT) that leverages Direct Preference Optimization (DPO) to rectify persistent translation errors. The approach utilizes a general text corpus and an expert translator (human or AI) to provide iterative feedback for preference-based learning. Applying this DPO-driven framework to the gemma3-1b model for English-to-German translation resulted in a substantial improvement in translation quality, increasing the COMET score from 0.703 to 0.747.
DPO-based post-training can significantly boost the translation quality of pre-trained NMT models like gemma3-1b, even without additional parallel data.
Contemporary neural machine translation (NMT) systems are almost exclusively built by training on supervised parallel data. Despite the tremendous progress achieved, these systems still exhibit persistent translation errors. This paper proposes that a post-training paradigm based on reinforcement learning (RL) can effectively rectify such mistakes. We introduce a novel framework that requires only a general text corpus and an expert translator which can be either human or an AI system to provide iterative feedback. In our experiments, we focus specifically on English-to-German translation as a representative high-resource language pair. Crucially, we implement this RL-based post-training using Direct Preference Optimization (DPO). Applying our DPO-driven framework to the gemma3-1b model yields a significant improvement in translation quality, elevating it's COMET score from 0.703 to 0.747 on the English to German task. The results demonstrate that DPO offers an efficient and stable pathway for enhancing pre-trained NMT models through preference-based post-training.
