Search papers, labs, and topics across Lattice.
This study investigates the self-preference bias in large language models (LLMs) by examining their responses to verified corrections of their own drafts. Using a deterministic verifier in the IFEval framework, the authors found that models, whether acting as their own authors or as neutral judges, rejected valid edits at nearly identical rates, indicating a lack of self-preference. The findings suggest that LLMs do not exhibit a bias towards their own outputs, as evidenced by the rejection rates being statistically indistinguishable between self-authored and fresh evaluations.
LLMs show no detectable self-preference, rejecting valid corrections to their own drafts at the same rate as neutral judges.
Large language models (LLMs) increasingly review and revise text, including their own. A documented self-preference bias (models favoring their own generations when acting as judges) raises the question of whether models also resist valid corrections to their own writing. We test this in a setting where "valid" is decided not by another model but by a deterministic verifier: instruction-following revision on IFEval. A model writes a draft; the official IFEval checker confirms the draft violates a constraint and that a candidate edit fixes it; the model then accepts or rejects that edit either as the genuine in-context author or as a fresh model that sees the draft neutrally. Across four mid-tier model families and 85 author-versus-fresh comparisons, we find no detectable self-preference: authors reject verified-good fixes to their own drafts at essentially the same rate as fresh models judging the same drafts (gap -5.1 pp, 95% CI [-12.9, +2.7]). A self-skepticism hint from a smaller pilot did not replicate at scale. The one robust observation is qualitative: when authors do reject a verified-good fix, 97% of their stated reasons are flaw-catching rather than preference, that is, about the character of rejections, not an elevated rate. Effects smaller than ~13 pp cannot be excluded at this sample size.
Tsinghua AI