Search papers, labs, and topics across Lattice.
This study investigates the impact of AI-assisted rewriting on clinical radiology reports, revealing significant information degradation through three rewriting tasks: EHR summarization, standardized rewriting, and teaching case preparation. The findings highlight a paradox where EHR summarization leads to the greatest loss of clinical entities and uncertainty language while maintaining cross-modal alignment, whereas tasks aimed at producing cleaner data actually degrade alignment more severely. The results emphasize that the type of rewriting task, rather than the clinical content itself, is the primary factor influencing information loss, raising critical implications for multimodal medical AI dataset construction.
EHR summarization erodes over half of clinical entities while preserving cross-modal alignment, revealing a surprising trade-off in AI-generated clinical documentation.
AI-assisted clinical documentation tools increasingly summarize, standardize, and reformat radiology reports using large language models (LLMs). We present a controlled measurement of the resulting information degradation. Using 450 chest X-ray reports from the Indiana University dataset, we generate synthetic versions via three realistic LLM rewriting tasks: EHR summarization, standardized rewriting, and teaching case preparation. We measure entity erosion (via medical NER), hedging collapse (loss of clinical uncertainty language), and cross-modal alignment degradation (via BiomedCLIP image-text similarity). Our central finding is a dissociation between information loss and cross-modal fidelity. EHR summarization is the most destructive at the content level, eroding 51.4% of clinical entities and 43.7% of hedging language, yet it preserves image-text alignment almost entirely (a 2.5% drop). The two tasks meant to produce cleaner training data, standardized rewriting and teaching case preparation, do the reverse: they preserve more entities (26.8% and 29.3% eroded) but cause 14.9-16.5% alignment drops, six to seven times those of EHR summarization. We term this the slop paradox: rewriting that makes clinical text look cleaner for multimodal training is precisely what pulls it away from the image. Contrary to our pre-specified hypothesis, rare pathologies were not preferentially degraded: across nine rare-versus-common comparisons, no difference survived multiple-comparison correction, and nominal differences ran in the opposite direction (common > rare), so contamination is invisible to condition-specific monitoring. The dominant determinant of degradation is the type of AI rewriting task, not the clinical content. These findings bear on multimodal medical AI dataset construction and the governance of AI-assisted clinical documentation.
