Search papers, labs, and topics across Lattice.
This paper introduces a knowledge-guided agentic framework designed to address patient-context ambiguity in healthcare chatbot queries by interpreting initial queries and generating targeted follow-up questions to gather essential patient-specific information. The framework utilizes a task-specific knowledge graph to construct plausible hypotheses and identify missing context variables, ultimately clarifying the prompt for downstream language models. Evaluation results show significant improvements in diagnosis retrieval accuracy and dietary-safety classification, with increases of at least 57.1 percentage points in Top-1 accuracy and 77.7 percentage points in Recall@5 across five language models compared to direct prompting.
By clarifying ambiguous patient queries, this framework boosts diagnostic accuracy by over 57 percentage points, transforming how healthcare chatbots interact with patients.
Patients often submit short, underspecified queries to healthcare chatbots that lack the patient-specific information needed to determine an appropriate response. Although these queries may be linguistically clear, they can support multiple plausible answers depending on undisclosed factors such as symptoms, diagnoses, medications, allergies, or dietary restrictions. A language model answering such a query directly may therefore rely on unsupported assumptions about the patient. We introduce a knowledge-guided agentic framework for mitigating patient-context ambiguity before final response generation. The framework operates between the patient and an otherwise unchanged downstream language model. It interprets the initial query, uses a task-specific knowledge graph to construct a set of plausible hypotheses, identifies the missing patient-context variables needed to distinguish among them, and asks targeted follow-up questions. The original query and the acquired context are then combined into a clarified prompt for the downstream model. We evaluated the framework across five language models using two controlled ambiguity-mitigation benchmarks: diagnosis retrieval from 1,034 symptom queries with clinically relevant evidence systematically masked, and dietary-safety classification from 487 queries with decisive health context omitted. The framework was compared with direct answering of the underspecified query and with rephrasing the same query without acquiring new patient information. In diagnosis retrieval, it increased overall exact Top-1 accuracy by at least 57.1 percentage points and selective exact Recall@5 by at least 77.7 percentage points across the five evaluated models compared with direct prompting. In dietary-safety classification, it improved accuracy across all five models and achieved the highest Matthews correlation coefficient for four...
