Search papers, labs, and topics across Lattice.
This paper introduces OKA-CT, a novel organ-hierarchical knowledge-augmented framework for medical vision-language pretraining that enhances the alignment between CT images and radiology reports. By leveraging a structured approach that incorporates organ-specific knowledge and fine-grained supervision, the model achieves superior performance in zero-shot abnormality diagnosis on the CT-RATE and RAD-ChestCT datasets. The results demonstrate a significant improvement in report-image alignment and sensitivity to disease-related anatomical regions, highlighting the importance of anatomical context in medical VLP tasks.
Organ-hierarchical representation learning boosts zero-shot abnormality diagnosis performance, achieving AUROCs of 84.9 and 72.2 on benchmark datasets.
Medical vision-language pretraining (VLP) from paired CT images and radiology reports enables scalable representation learning, but most existing methods align either whole scans with entire reports or local image regions with text fragments. These formulations underuse a key property of radiology reports: findings are organized around anatomical structures, with abnormalities described by organs, disease concepts, locations, and severity-related attributes. We propose OKA-CT, an organ-hierarchical knowledge-augmented framework for CT-report VLP. OKA-CT first converts free-text reports into organ-conditioned knowledge using radiology report parsing and LLM-assisted semantic structuring. The extracted hierarchy is used across two learning stages. Stage~1 injects anatomy-grounded evidence into the CT visual representation through fine-grained organ-conditioned supervision, while Stage~2 uses organ-specific report evidence to guide structured report-CT contrastive learning, where hierarchy-derived semantic soft targets treat non-paired cases with shared organ-level findings as weak semantic positives rather than uniform negatives. A lightweight query-based global branch further aggregates disease-relevant volumetric evidence for whole-scan representation. On CT-RATE and RAD-ChestCT datasets, OKA-CT achieves zero-shot abnormality diagnosis AUROCs of 84.9 and 72.2, outperforming prior CT VLP baselines. Retrieval and patch-occlusion analyses further show improved report-image alignment and stronger sensitivity to disease-associated anatomical regions.
