Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the issue of metric overfitting in Minimum Bayes Risk (MBR) decoding, which can lead to inflated utility metrics that compromise other evaluation criteria. The authors introduce SVD-MBR, a method that utilizes Singular Value Decomposition to create a low-rank approximation of the pairwise utility matrix, effectively filtering out metric noise. Experimental results show that SVD-MBR not only regularizes decoding but also enhances performance across various generalized metrics, revealing that the effectiveness of this approach is dependent on the type of neural metrics used.
SVD-MBR reveals that by filtering out metric noise, we can significantly improve text generation quality while avoiding the pitfalls of overfitting.
Minimum Bayes Risk (MBR) decoding enables high-quality text generation by selecting the hypothesis that maximizes a utility metric over sampled pseudo-references. However, it is highly susceptible to metric overfitting: it can irregularly inflate the chosen utility metric at the direct expense of other unoptimized evaluation metrics. To mitigate this, we introduce SVD-MBR, which frames the pairwise utility matrix as a noisy information signal. By computing a low-rank approximation via Singular Value Decomposition (SVD) and retaining only the top-$k$ components, we effectively decouple true consensus from metric noise. Experiments demonstrate that SVD-MBR successfully regularizes decoding, yielding substantial gains across a range of generalized metrics. Furthermore, we reveal that this denoising is metric-dependent: neural metrics encode a robust low-rank consensus ideal for SVD, whereas surface-level metrics struggle to separate signal from metric noise.