Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates how misleading context affects the medical judgment of large language models (LLMs) in question-answering tasks. By injecting fabricated evidence and bare assertions into medical queries from the MedMisBench dataset, the authors reveal that models are significantly more influenced by assertions, adopting incorrect answers more frequently. The findings highlight that while misleading cues are often disclosed in reasoning traces, they are less frequently reflected in the final responses, indicating a critical gap in model transparency and reliability.
Misleading assertions corrupt LLM medical reasoning more than fabricated evidence, yet are disclosed less often, exposing a critical flaw in model transparency.
Large language models now answer medical questions with expert-level performance. However, the context these systems act on can be misleading, and misleading context can corrupt a model's medical judgment. To understand how misleading context corrupts this judgment, we examine the model's susceptibility to the context, disclosure of it, mechanism of corrupted reasoning, and monitorability of the decision. On the medical reasoning subset of MedMisBench, a clinician-reviewed question-answering benchmark of 8,627 questions, we inject two types of misleading context cues, fabricated evidence and a bare assertion. We test three reasoning models, two that expose their full reasoning trace and one frontier model that exposes only its response. All three are more susceptible to the assertion than to the fabricated evidence, adopting the asserted answer 10 to 27 points more often. The misleading cues are disclosed in 81 to 98% of traces but only 7 to 90% of responses, and the assertion is disclosed less often than evidence based cues. Resampling from reasoning traces without disclosure shows the two cues corrupt reasoning differently, evidence entering early and accumulating while the assertion redirects the conclusion near its end. An LLM monitor catches 78% of corrupted decisions at 5% false positives when reading an open model's trace with guidance, against at most 32% from any response. The misleading context that models are most susceptible to is disclosed least, and was caught reliably only from an open reasoning trace, which frontier providers withhold.