Search papers, labs, and topics across Lattice.
This paper addresses the limitations of Minimum Bayes Risk (MBR) decoding by introducing a noisy channel decomposition that accounts for bidirectional effects in hypothesis selection. The authors demonstrate that traditional evaluation metrics like BLEU and COMET exhibit asymmetries that can misrepresent the quality of generated text, and their method allows for a more nuanced understanding of these effects. Their analysis shows that channel-wise contributions vary across metrics but are stable across tasks, suggesting that optimized channel weighting could enhance MBR decoding performance.
Channel-wise contributions in MBR decoding reveal distinct characteristics that could significantly improve text generation quality when properly weighted.
Minimum Bayes Risk (MBR) decoding yields more robust and higher-quality text generation than maximum a posteriori (MAP) decoding by selecting hypotheses that maximize expected utility over sampled pseudo-references. However, there exists a discrepancy in the design: hypothesis selection calculates expected utility scores conditioned on given pseudo-references, while commonly used evaluation metrics, e.g., BLEU and COMET, are asymmetric. Therefore, it is important to consider both hypothesis-to-reference and reference-to-hypothesis directional effects. In this study, we introduce a noisy channel decomposition of MBR decoding that naturally incorporates bidirectional effects to account for these asymmetries. We decompose MBR decoding into four interacting components: hypothesis-to-reference likelihood, reference-to-hypothesis likelihood, hypothesis prior, and reference prior. This decomposition provides a unified interpretation of existing MBR variants and enables metric- and task-specific interpretability by isolating the contribution of each channel. Our comprehensive analysis reveals that channel-wise contributions exhibit distinct characteristics across metrics while remaining consistent across tasks, and suggests that appropriate channel weighting may lead to improvements over original MBR decoding.
