Search papers, labs, and topics across Lattice.
This paper introduces a two-stage framework for medical hypothesis verification that addresses the reliability of large language models (LLMs) by allowing them to abstain from uncertain predictions, thereby improving accuracy. The framework strategically employs ontology grounding only when the model abstains, effectively managing the tradeoff between coverage and accuracy. Experiments with GPT-5.5 and DeepSeek-R1 demonstrate significant improvements in question-level accuracy (from 82.9% to 92.5%) and hypothesis-level accuracy (from 92.0% to 96.2%) on medical datasets, highlighting the potential of abstention as a control signal for selective reasoning refinement.
Abstaining from uncertain predictions can enhance LLM accuracy in medical applications by 9.6 percentage points, transforming uncertainty into a strategic advantage.
Large language models (LLMs) often rely on shortcuts rather than systematic reasoning, raising safety concerns in medical applications. Allowing models to abstain when uncertain improves reliability but introduces a coverage accuracy tradeoff. We propose a two-stage framework for medical hypothesis verification in multiple-choice settings that manages this tradeoff through targeted ontology grounding, applied only when the model abstains. We show that abstention is not random but reflects genuine uncertainty, with abstained predictions associated with lower confidence. Across two frontier models (GPT-5.5, accessed via the Azure OpenAI API, and DeepSeek-R1), the proposed framework improves question-level accuracy by 9.6 percentage points (82.9% to 92.5%) and hypothesis-level accuracy by 4.2 percentage points (92.0% to 96.2%). Our experiments conducted on MedReason and MedQA show that abstention can be repurposed as a control signal for selective reasoning refinement, achieving knowledge-graph-level performance without explicit knowledge graph construction.
Microsoft Research