Search papers, labs, and topics across Lattice.
The paper introduces MIMIC-DOS, a new dataset derived from MIMIC-IV, designed to evaluate LLMs' ability to handle conflicting signs and symptoms for organ dysfunction prediction in ICU settings. To address the challenges posed by evidence discordance, they propose CARE, a privacy-compliant agentic reasoning framework that separates guidance generation (remote LLM) from evidence processing and decision-making (local LLM). Experiments demonstrate that CARE outperforms baselines on MIMIC-DOS, showcasing its ability to handle conflicting clinical evidence while preserving patient privacy.
LLMs struggle with conflicting medical evidence, but a clever two-stage agentic approach can reconcile discordant signals while preserving patient privacy.
Large language model (LLM) systems are increasingly used to support high-stakes decision-making, but they typically perform worse when the available evidence is internally inconsistent. Such a scenario exists in real-world healthcare settings, with patient-reported symptoms contradicting medical signs. To study this problem, we introduce MIMIC-DOS, a dataset for short-horizon organ dysfunction worsening prediction in the intensive care unit (ICU) setting. We derive this dataset from the widely recognized MIMIC-IV, a publicly available electronic health record dataset, and construct it exclusively from cases in which discordance between signs and symptoms exists. This setting poses a substantial challenge for existing LLM-based approaches, with single-pass LLMs and agentic pipelines often struggling to reconcile such conflicting signals. To address this problem, we propose CARE: a multi-stage privacy-compliant agentic reasoning framework in which a remote LLM provides guidance by generating structured categories and transitions without accessing sensitive patient data, while a local LLM uses these categories and transitions to support evidence acquisition and final decision-making. Empirically, CARE achieves stronger performance across all key metrics compared to multiple baseline settings, showing that CARE can more robustly handle conflicting clinical evidence while preserving privacy.
