Search papers, labs, and topics across Lattice.
The paper introduces PassiveQA, a three-action framework (Answer, Ask, Abstain) to improve epistemic awareness in LLMs for question answering, addressing the issue of overconfident or hallucinated responses to underspecified queries. They finetune a planner using structured information-state representations and knowledge graph-grounded context to explicitly model missing variables and decision reasoning. Results across multiple QA datasets demonstrate that the finetuned planner significantly improves macro F1 and abstention recall while reducing hallucination rates, showing that epistemic decision-making must be learned during training.
LLMs struggle to recognize when they lack sufficient information to answer a question, but a surprisingly simple finetuning approach can teach them to abstain or ask for clarification, dramatically reducing hallucinations.
Large Language Models (LLMs) have achieved strong performance in question answering and retrieval-augmented generation (RAG), yet they implicitly assume that user queries are fully specified and answerable. In real-world settings, queries are often incomplete, ambiguous, or missing critical variables, leading models to produce overconfident or hallucinated responses. In this work, we study decision-aware query resolution under incomplete information, where a model must determine whether to Answer, Ask for clarification, or Abstain. We show that standard and enhanced RAG systems do not reliably exhibit such epistemic awareness, defaulting to answer generation even when information is insufficient. To address this, we propose PassiveQA, a three-action framework that aligns model behaviour with information sufficiency through supervised finetuning. Our approach integrates structured information-state representations, knowledge graph-grounded context, and a finetuned planner that explicitly models missing variables and decision reasoning. Experiments across multiple QA datasets show that the finetuned planner achieves significant improvements in macro F1 and abstention recall while reducing hallucination rates, under a compute-constrained training regime. These results provide strong empirical evidence that epistemic decision-making must be learned during training rather than imposed at inference time.
