Search papers, labs, and topics across Lattice.
This paper introduces Anatomy Contextualized Adaptation (ACA), a novel framework that enhances the performance of CT vision-language foundation models by aligning fine-grained anatomical features with global context. ACA utilizes TotalSegmentator to generate anatomy-level embeddings, which are then refined through a transformer that captures inter-anatomy relationships and aligns with radiology report text. Evaluations on the Merlin and CT-RATE datasets show that ACA outperforms existing methods in zero-shot classification while significantly reducing training time to under one hour, demonstrating its efficiency and effectiveness in anatomical context adaptation.
ACA achieves superior zero-shot classification performance in CT imaging by balancing fine-grained anatomical alignment with essential global context, all in under an hour of training.
CT vision-language foundation models have demonstrated promising performance across downstream tasks, but are typically trained with whole-volume representations that dilute fine-grained anatomical signals. Fine-grained vision-language pre-training addresses this by aligning anatomy-level visual features with anatomy-specific text, but in doing so discards the global context that whole-volume models provide. Furthermore, existing fine-grained approaches train from scratch, making them computationally expensive. We introduce Anatomy Contextualized Adaptation (ACA), a lightweight framework that adapts frozen CT foundation model representations for anatomy-level vision-language alignment while enhancing global contextualization. ACA uses TotalSegmentator to decompose CT volumes into anatomy-level embeddings, which are refined via a transformer that captures cross-anatomy relationships, and aligned to both per-anatomy and scan-level text extracted from radiology reports. Evaluated on Merlin and CT-RATE, ACA consistently outperforms both the frozen foundation model baselines and existing fine-grained methods in zero-shot finding classification, while requiring less than one hour of training once embeddings are cached. The attention weights learned by ACA's inter-anatomy transformer additionally indicate plausible cross-anatomy context routing. Altogether, these results support ACA as a lightweight approach for adapting CT foundation models to anatomically grounded vision-language alignment while preserving and enhancing global anatomical context.
