Search papers, labs, and topics across Lattice.
This paper introduces Conformal Abstention (CA), a post-hoc framework for language models that determines when to abstain from answering a query based on prediction confidence, providing finite-sample guarantees on participation and correctness. To improve the alignment between prediction confidence and actual knowledge, they propose a calibration strategy leveraging representation geometry to measure knowledge involvement. Experiments show that CA significantly improves selective answering, achieving 75% conditional correctness.
LMs can now selectively abstain from answering with provable guarantees, thanks to a new method that uses representation geometry to better gauge when they're out of their depth.
When language models lack relevant knowledge for a given query, they frequently generate plausible responses that can be hallucinations, rather than admitting being agnostic about the answer. Retraining models to reward admitting ignorance can lead to overly conservative behaviors and poor generalization due to scarce evaluation benchmarks. We propose a post hoc framework, Conformal Abstention (CA), adapted from conformal prediction (CP) to determine whether to abstain from answering a query. CA provides finite-sample guarantees on both the probability of participation (i.e., not abstaining) and the probability that the generated response is correct. Importantly, the abstention decision relies on prediction confidence rather than the non-conformity scores used in CP, which are intractable for open-ended generation. To better align prediction confidence with the model's ignorance, we introduce a calibration strategy using representation geometry within the model to measure knowledge involvement in shaping the response. Experiments demonstrate that we improve selective answering significantly with 75 percent conditional correctness.
