Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a preference learning approach to improve factual consistency in summarization by aggregating multiple imperfect factuality metrics. They generate summary pairs with subtle lexical differences and filter out instances with high metric disagreement to create a high-quality preference dataset. Experiments show consistent factuality improvements across various models, including LLMs, with smaller models achieving comparable factuality to larger ones.
Combining imperfect metrics and learning from subtle lexical variations lets smaller summarization models punch above their weight class in factual consistency.
Reinforcement learning with evaluation metrics as rewards is widely used to enhance specific capabilities of language models. However, for tasks such as factually consistent summarisation, existing metrics remain underdeveloped, limiting their effectiveness as signals for shaping model behaviour.While individual factuality metrics are unreliable, their combination can more effectively capture diverse factual errors. We leverage this insight to introduce an automated training pipeline that improves factual consistency in summaries by aggregating scores from different weak metrics. Our approach avoids the need for complex reward shaping by mapping scores to preferences and filtering out cases with high disagreement between metrics. For each source document, we generate lexically similar summary pairs by varying decoding strategies, enabling the model to learn from factual differences caused by subtle lexical differences. This approach constructs a high-quality preference dataset using only source documents.Experiments demonstrate consistent factuality gains across models, ranging from early encoder-decoder architectures to modern large language models, with smaller models reaching comparable factuality to larger ones.