Search papers, labs, and topics across Lattice.
This study investigates the reliability of LLM judges in evaluating open-ended responses without reference answers, employing a two-stage pipeline that includes calibration and sensitivity experiments across three languages. The findings reveal that these judges tend to over-credit incorrect answers when no reference is provided, with the introduction of reference information significantly altering their evaluations鈥攗p to 85% in some cases. The results highlight the necessity of calibrating LLM judges with reference-aware evaluations to ensure their effectiveness in reference-free contexts, aligning closely with human judgment in many instances.
LLM judges can drastically change their evaluations鈥攂y up to 85%鈥攚hen provided with reference answers, revealing a critical flaw in no-reference assessments.
LLM judges are increasingly being used to evaluate open-ended model responses, often in no-reference settings where a ground-truth answer is unavailable. However, can they reliably assess in such evaluation setups? We explore this question in this paper through a two stage pipeline with a) calibration experiments that assess the judge model's knowledge of the task it is evaluating, and b) sensitivity experiments that assess how the judge model's performance is impacted by the presence and positioning of the reference answer in the prompt. Across experiments covering three languages, we show that the judge models we evaluated tend to over-credit incorrect answers in the absence of a reference answer, and adding reference answer information to the prompt flips the judge model's correct/incorrect decisions by as much as 85% in some experimental settings. Comparison with a subset of human annotations shows that these reference-driven changes generally align with human judgments. Our results emphasize the need for calibrating the LLM judges with a sample with reference-aware evaluation before using them in reference-free setups reliably, and our methodology provides a blueprint for researchers and practitioners in doing such calibration of LLM judges for other tasks.
