Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors introduce CLIMB, a benchmark evaluating multi-turn doctor-patient conversations for multi-label diagnosis of co-occurring clinical conditions synthesized from structured decision algorithms. Testing across six frontier and open models reveals severe failure, with no model recovering the exact ground-truth set of conditions in more than 10% of interactive consultations. Crucially, the degradation is driven by "single-hypothesis tracking," where models anchor on initial findings, constrain follow-up inquiries to that narrow hypothesis, and generate spurious diagnoses rather than discovering secondary conditions.
Frontier LLMs fail to diagnose co-occurring medical conditions in over 90% of interactive consultations because they suffer from single-hypothesis fixation—anchoring on initial findings and failing to explore parallel diagnostic branches.
Patients often have several co-occurring clinical conditions, and the findings needed to identify and disambiguate them emerge over the course of a consultation. Evaluating clinical reasoning in this setting requires both multi-turn interaction and multi-label diagnosis. We introduce CLIMB, a benchmark in which a doctor model interviews a simulated patient to recover a ground truth set of co-occurring clinical conditions. Cases are synthesized from clinical decision algorithms and diagnostic datasets, grounding multimorbid presentations in structured clinical knowledge. Across six frontier and open models, none recovers the exact set of conditions in more than 10% of interactive cases. Diagnostic performance declines when conditions co-occur, even when models receive the full clinical record and the true number of conditions. Interaction reduces performance further. In controlled experiments, models behave like single-hypothesis trackers: they anchor on the diagnosis suggested by the opening findings, keep questioning around it, and recover a second condition mainly when a finding in view points to it. Questioning them further does not complete the set but adds mostly wrong diagnoses. We formalise this pattern with a theoretical reference model of single-hypothesis tracking. The benchmark, generator, and evaluation code are available at https://anonymous.4open.science/r/CLIMB-8340.