Search papers, labs, and topics across Lattice.
This study introduces PerFact, a novel approach for generating radiology reports from 3D brain MRI by leveraging perception-derived facts rather than relying solely on model architecture or size. The researchers found that traditional zero-shot models and fine-tuned backbones performed poorly, indicating that the quality of the generated reports is primarily determined by the information injected into the prompt. By structuring outputs from 3D segmentation and classification into fact sentences, PerFact significantly improves report quality, demonstrating that the grounding information is more critical than the choice of model architecture.
Grounding information, not model architecture, is the key to generating high-quality radiology reports from 3D brain MRI scans.
Radiology report generation has matured almost entirely on 2D chest radiographs, where the default route to better reports is a larger backbone or a pre-training one on medical data. We revisit that assumption on 3D multi-sequence brain MRI, a volumetric multi-disease regime, and find that the model is not the lever. Zero-shot medical and radiology vision-language models transfer poorly to brain MRI, with chest radiograph specialists failing most conspicuously, and five backbones fine-tuned identically across three model families and an order of magnitude in scale differ only marginally. What determines the quality of the report is the information injected into the prompt. We delegate perception to upstream 3D segmentation and classification, serialize their outputs into a structured fact sentence, and prompt a LoRA-adapted vision-language model with it; we call this \textbf{PerFact}. In a controlled study that fixes the backbone, data split, target reports, and adaptation while varying only the injected grounding, perception-derived facts outperform retrieved prior reports, retrieval becomes redundant once facts are present, and end-to-end predicted facts remain effective without any ground-truth annotation at inference. The residual gap between predicted and oracle facts is explained by the granularity of the facts rather than by the generator. Closed-ended visual question answering comes at no measurable cost to report quality, though the grounding source has little effect on it. On 3D brain MRI, grounding information, not model choice, is the dominant controllable factor in report quality.
