Search papers, labs, and topics across Lattice.
The paper introduces I-CALM, a prompt-based framework that incentivizes LLMs to abstain from answering factual questions when uncertain, aiming to mitigate hallucination. I-CALM elicits verbal confidence scores, rewards abstention, and incorporates normative principles promoting truthfulness and humility. Experiments using GPT-3.5 mini on PopQA demonstrate that I-CALM reduces the false-answer rate by shifting error-prone cases to abstention, effectively trading coverage for reliability without retraining the model.
LLMs can be coaxed into admitting "I don't know" more often, slashing hallucination rates, simply by tweaking prompts to reward humility and truthfulness.
Large language models (LLMs) frequently produce confident but incorrect answers, partly because common binary scoring conventions reward answering over honestly expressing uncertainty. We study whether prompt-only interventions -- explicitly announcing reward schemes for answer-versus-abstain decisions plus humility-oriented normative principles -- can reduce hallucination risk without modifying the model. Our focus is epistemic abstention on factual questions with a verifiable answer, where current LLMs often fail to abstain despite being uncertain about their answers. We first assess self-reported verbal confidence as a usable uncertainty signal, showing stability under prompt paraphrasing and reasonable calibration against a token-probability baseline. We then study I-CALM, a prompt-based framework that (i) elicits verbal confidence, (ii) partially rewards abstention through explicit reward schemes, and (iii) adds lightweight normative principles emphasizing truthfulness, humility, and responsibility. Using GPT-5 mini on PopQA as the main setting, we find that confidence-eliciting, abstention-rewarding prompts, especially with norms, reduce the false-answer rate on answered cases mainly by identifying and shifting error-prone cases to abstention and re-calibrating their confidence. This trades coverage for reliability while leaving forced-answer performance largely unchanged. Varying the abstention reward yields a clear abstention-hallucination frontier. Overall, results show the framework can improve selective answering on factual questions without retraining, with the magnitude of effect varying across models and datasets. Code is available at the following https://github.com/binzeli/hallucinationControl.
