Search papers, labs, and topics across Lattice.
This study investigates the calibration of instruction-tuned large language models (LLMs) in conversational settings, revealing that these models exhibit significant overconfidence in their own responses due to an "ownership bias." By analyzing six recent open-weight LLMs, the authors demonstrate that models assign up to 26% higher confidence to their own answers compared to identical user-provided answers. To mitigate this issue, the authors propose a novel inference-time strategy that rephrases model outputs as user inputs during confidence elicitation, leading to improved calibration without retraining.
Instruction-tuned LLMs can be up to 26% overconfident in their own responses, but a simple adjustment can significantly improve their calibration.
Prior work has shown that instruction-tuned large language models (LLMs) are less well calibrated than their base pre-trained counterparts. However, little is known about the frequently used chat template's effect on the calibration of conversational LLMs. In this work, we investigate the mechanisms driving this miscalibration by decoupling the effects of the post-training algorithm and the chat format. We find that, while instruction tuning fundamentally harms calibration, the chat template aggravates the issue through an "ownership bias" -- models are significantly more confident in their own answers than in identical answers provided by a user. Extensive experiments across six recent open-weight LLMs, three benchmarks, and three confidence elicitation methods show that models assign up to 26% higher confidence to their own responses. Leveraging this insight, we propose a simple inference-time strategy: framing the model's answer as user input during confidence elicitation. This approach significantly reduces overconfidence and improves calibration by up to 26% without the need for retraining, narrowing the gap between base and instruction-tuned models.
