Search papers, labs, and topics across Lattice.
The paper introduces Concept-Enhanced Multimodal RAG (CEMRAG), a novel framework for Radiology Report Generation (RRG) that integrates concept-based explainability with multimodal Retrieval-Augmented Generation (RAG). CEMRAG decomposes visual representations into clinical concepts and uses these concepts to enrich contextual prompts for VLMs, aiming to improve both interpretability and factual accuracy in generated reports. Experiments on MIMIC-CXR and IU X-Ray datasets demonstrate that CEMRAG outperforms conventional RAG and concept-only baselines across various VLM architectures, training regimes, and retrieval configurations.
Radiology report generation can be both more accurate AND more interpretable: CEMRAG uses visual concepts to enhance multimodal RAG, challenging the assumed trade-off between transparency and diagnostic accuracy.
Radiology Report Generation (RRG) through Vision-Language Models (VLMs) promises to reduce documentation burden, improve reporting consistency, and accelerate clinical workflows. However, their clinical adoption remains limited by the lack of interpretability and the tendency to hallucinate findings misaligned with imaging evidence. Existing research typically treats interpretability and accuracy as separate objectives, with concept-based explainability techniques focusing primarily on transparency, while Retrieval-Augmented Generation (RAG) methods targeting factual grounding through external retrieval. We present Concept-Enhanced Multimodal RAG (CEMRAG), a unified framework that decomposes visual representations into interpretable clinical concepts and integrates them with multimodal RAG. This approach exploits enriched contextual prompts for RRG, improving both interpretability and factual accuracy. Experiments on MIMIC-CXR and IU X-Ray across multiple VLM architectures, training regimes, and retrieval configurations demonstrate consistent improvements over both conventional RAG and concept-only baselines on clinical accuracy metrics and standard NLP measures. These results challenge the assumed trade-off between interpretability and performance, showing that transparent visual concepts can enhance rather than compromise diagnostic accuracy in medical VLMs. Our modular design decomposes interpretability into visual transparency and structured language model conditioning, providing a principled pathway toward clinically trustworthy AI-assisted radiology.
