Search papers, labs, and topics across Lattice.
The paper introduces Second Guess, a parameter-free prompting technique for uncertainty detection in small language models (SLMs) for multiple-choice question answering. It leverages the insight that confident models exhibit stable answer selection, while uncertain models show instability when presented with an "I don't know" option. Experiments on 2B-8B parameter models across four benchmarks demonstrate a 10.81% improvement in composite risk compared to existing methods, even outperforming entropy-based methods on fine-tuned models.
Small language models can reliably signal uncertainty without any parameter updates, simply by observing how consistently they answer when given the option to say "I don't know."
Large language models often generate confident but incorrect answers rather than abstaining when uncertain. This problem is particularly acute for small language models (SLMs), where computational constraints and autonomous operation amplify the need for reliable uncertainty detection. We propose _Second Guess_, a lightweight, parameter-free prompting technique for abstention in multiple-choice question answering (MCQA) that is well-suited for SLMs. Our key empirical insight is that models which truly know an answer will select it consistently, while uncertain models exhibit unstable behavior when an ``I don't know'' option is added. Evaluated on four open models (2B-8B parameters) and four benchmarks, Second Guess achieves the highest composite risk improvement of 10.81\%. Notably, it maintains an 8\% composite risk improvement on fine-tuned models where entropy-based methods degrade, and improves most for lower-performing models. All code and results required to reproduce this work is available in https://github.com/Mystic-Slice/second-guess
