Search papers, labs, and topics across Lattice.
This paper introduces STAR3, a multimodal framework designed for fine-grained radiology report retrieval that integrates spatio-temporal and clinical conditioning. By leveraging an object detector for anatomical region identification and incorporating longitudinal disease progression, STAR3 significantly enhances the relevance of retrieved report sentences in the context of current clinical indications. Experimental results on the MIMIC-CXR dataset show that STAR3 outperforms existing retrieval-based methods across various metrics, underscoring its potential to improve automated radiology reporting practices.
STAR3 redefines automated radiology report generation by seamlessly integrating anatomical, temporal, and clinical context, leading to more relevant and accurate report retrieval.
Radiology is vital to modern healthcare, but rising imaging demand and persistent workforce shortages strain reporting capacity and clinical workflows. Automated radiology report generation has the potential to support radiologists and help alleviate this burden; however, existing retrieval-based methods remain rigid, lack explicit anatomical grounding, and do not account for longitudinal disease progression or available clinical context. In this work, we introduce STAR3, a multimodal, spatio-temporal, attentive retrieval framework for radiology report generation that aligns region-level anatomical information with clinical indications and longitudinal changes across chest X-ray studies. Our framework employs an object detector to identify anatomically meaningful regions and retrieves semantically relevant report sentences conditioned on both current clinical context and changes observed between prior and current examinations. This design enables anatomically and temporally grounded report generation that better reflects clinical reporting practice. Experiments on the MIMIC-CXR dataset demonstrate that STAR3 outperforms current retrieval-based approaches on retrieval, NLP and clinical metrics, highlighting the value of conditioning retrieval anatomically, temporally and clinically for advancing automated radiology report generation.
