Search papers, labs, and topics across Lattice.
The paper introduces Group Relative Reward Modeling (GRRM) to improve intra-group ranking within Group Relative Policy Optimization (GRPO) for machine translation, addressing the limitations of Scalar Quality Metrics (SQM) that evaluate candidates in isolation. GRRM employs a Group Quality Metric (GQM) paradigm, processing entire candidate groups jointly to leverage comparative analysis for more accurate relative quality assessment. Experiments demonstrate that GRRM enhances translation quality and unlocks reasoning capabilities when integrated into the GRPO training loop.
Machine translation gets a boost: a new reward model leverages comparative analysis of candidate translations to unlock reasoning abilities comparable to SOTA reasoning models.
While Group Relative Policy Optimization (GRPO) offers a powerful framework for LLM post-training, its effectiveness in open-ended domains like Machine Translation hinges on accurate intra-group ranking. We identify that standard Scalar Quality Metrics (SQM) fall short in this context; by evaluating candidates in isolation, they lack the comparative context necessary to distinguish fine-grained linguistic nuances. To address this, we introduce the Group Quality Metric (GQM) paradigm and instantiate it via the Group Relative Reward Model (GRRM). Unlike traditional independent scorers, GRRM processes the entire candidate group jointly, leveraging comparative analysis to rigorously resolve relative quality and adaptive granularity. Empirical evaluations confirm that GRRM achieves competitive ranking accuracy among all baselines. Building on this foundation, we integrate GRRM into the GRPO training loop to optimize the translation policy. Experimental results demonstrate that our framework not only improves general translation quality but also unlocks reasoning capabilities comparable to state-of-the-art reasoning models. We release codes, datasets, and model checkpoints at https://github.com/NJUNLP/GRRM.
