Search papers, labs, and topics across Lattice.
This study identifies and quantifies a phenomenon termed Narrative Anchoring, where large language models (LLMs) produce divergent diagnostic outputs based solely on the sociolinguistic register of identical clinical facts. By constructing a dataset of 1,000 USMLE clinical vignettes rewritten in three distinct registers, the authors demonstrate that this bias is statistically significant across multiple model architectures, with a Narrative Anchoring Gap ranging from 0.064 to 0.151. The introduction of NarrativeShield, a three-agent pipeline for fact extraction and verification, effectively mitigates this bias, reducing the gap to near-zero while maintaining a low rate of unstable decisions, albeit with a slight accuracy trade-off.
Narrative Anchoring reveals that language models can misdiagnose identical clinical facts based solely on their sociolinguistic presentation, leading to significant diagnostic variability.
Large language models used for clinical diagnostic reasoning are sensitive to sociolinguistic register, not just clinical content. We term this failure mode Narrative Anchoring: identical clinical facts expressed in different registers cause diagnostic outputs to diverge. Unlike prior demographic-bias work, which manipulates explicit identity tokens such as race or income, our benchmark isolates register as the sole channel of variation, with no demographic marker present in any form. We construct a dataset of 1,000 USMLE clinical vignettes, each rewritten into three sociolinguistically distinct personas under an independently audited fact-preservation guarantee, verified by a separate model that never sees the generation prompt. Across seven language models spanning three architecture families and scales, Narrative Anchoring is statistically significant under direct prompting in every model tested, with a Narrative Anchoring Gap of 0.064 to 0.151. Chain-of-thought reasoning and explicit debiasing instructions reduce the bias only partially, and their apparent gains are frequently confounded by accuracy collapse. We introduce NarrativeShield, a three-agent pipeline that structurally extracts and verifies clinical facts before diagnostic reasoning begins, reducing the Narrative Anchoring Gap to near-zero ($-0.004$ to $0.037$) and achieving the lowest rate of severely unstable decisions (DSS $<$ 0.8) of any method across all models, at a modest and mechanistically expected accuracy cost for most models. A stress test using a non-instruction-tuned base model shows that executing a debiasing intervention at all is gated by zero-shot instruction-following ability, not prompt content alone. We release our dataset, human-validated for fact preservation, as a standalone resource for studying register-based clinical bias.
