Search papers, labs, and topics across Lattice.
This paper investigates how the presence and quality of reasoning chains influence LLM-based judges in factual QA and mathematical reasoning tasks. They find that weaker LLM judges are easily misled by fluent reasoning, even when the answer is incorrect, while stronger judges can leverage reasoning to some extent. However, even strong judges are susceptible to being misled by seemingly high-quality but ultimately flawed reasoning chains, with both fluency and factuality of reasoning being key drivers of the judge's decision.
Even the strongest LLM judges can be easily fooled by seemingly high-quality reasoning chains, highlighting a critical vulnerability in using LLMs to evaluate other LLMs.
Large language models (LLMs) has been widely adopted as a scalable surrogate for human evaluation, yet such judges remain imperfect and susceptible to surface-level biases. One possible reason is that these judges lack sufficient information in assessing answer correctness. With the rise of reasoning-capable models, exposing a generator's reasoning content to the judge provides richer information and is a natural candidate for improving judgment accuracy. However, its actual impact on judge behavior remains understudied. In this paper, we systematically investigate how access to reasoning chains affects LLM-based judgment across factual question answering (QA) and mathematical reasoning benchmarks. We find that weak judges are easily swayed by reasoning presence, frequently accepting incorrect answers accompanied by fluent reasoning, while strong judges can partially leverage reasoning as informative evidence. Nevertheless, even strong judges are misled by seemingly high-quality reasoning chains. Controlled experiments further reveal that both fluency and factuality of reasoning chains are critical signals driving judge decisions. These findings highlight the need for more robust LLM judges that can distinguish genuine reasoning quality from superficial fluency when evaluating modern reasoning models.
