Search papers, labs, and topics across Lattice.
This paper investigates the latent ability of large language models (LLMs) to predict their own output quality as judged by external evaluators, finding that base models can achieve above-chance performance in this task without targeted training. The authors introduce Self-Evaluation Elicitation (SEE), a novel method that combines calibration-coupled reinforcement learning with masked distillation to enhance the model's self-evaluation capabilities while maintaining output quality. Results demonstrate that SEE significantly improves calibration with 31 times fewer examples compared to traditional reinforcement learning methods, suggesting a robust and transferable understanding of quality within the model's token distribution.
Base LLMs can predict their own output quality with surprising accuracy, revealing a latent self-evaluation capability that can be harnessed with minimal data.
Large language models are increasingly evaluated by other models, raising a natural question: can a model predict how a judge will score its own output? We find that the ability is largely present before any targeted training: prompted few-shot, a base model already predicts an external judge's multi-attribute quality scores on open-ended responses well above chance across three benchmarks. We introduce Self-Evaluation Elicitation (SEE), a method that surfaces this latent ability through a short cycle comprising a calibration-coupled reinforcement learning phase that improves the answer and predicts the judge, followed by a masked distillation phase that sharpens the prediction while leaving the answer untouched. From 160 unique examples, roughly 31x fewer than a reinforcement learning baseline, SEE improves held-out calibration across three benchmarks while preserving answer quality. The elicited self-evaluation is sharply localized within the model's own token distribution and stable across judges it was never trained against, indicating a transferable notion of quality rather than a single judge's preference. These results reframe judge-aligned self-evaluation as a problem of elicitation rather than acquisition.
NUS