Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a novel label-free selection criterion for foundational models in medical image classification, addressing the challenge of performance variability under inter-institutional distribution shifts. Utilizing the SUDO framework, the authors measure pseudo-label discrepancies across unlabeled target data to derive an AURCC score that effectively ranks various vision-language models without requiring target-domain annotations. The results demonstrate that AURCC can accurately recover ground-truth rankings, achieving a Spearman correlation of up to 0.943, particularly excelling in scenarios with limited labeled data.
AURCC can rank foundational models for medical image classification with remarkable accuracy, even in the absence of target-domain labels.
Foundation models are increasingly deployed for medical image analysis. However, under the inter-institutional distribution shift typical of deployment, their performance varies widely and cannot be known without target-domain labels, which are rarely available. This leaves a practical question unresolved: given several candidate foundational models and labeled-data from a source domain, which one to deploy in an unlabeled target domain? We propose a label-free selection criterion built on SUDO, a framework for evaluating clinical AI systems without ground-truth annotations. SUDO partitions the unlabeled target data by predicted probability and, for each region, measures a pseudo-label discrepancy reflecting class contamination; aggregated across regions, this yields a score (AURCC) requiring neither target annotation nor fine-tuning. We show that AURCC can be used to rank a variety of vision-language models (BioMedCLIP, CXR-CLIP, CheXzero, MedCLIP, MedImageInsight, CLIP) on chest X-ray classification across three inter-hospital shift scenarios, under zero-shot and MLP-probe regimes. The AURCC ranking recovers the ground-truth ranking with Spearman rho up to 0.943 (p<0.05). Against the natural baseline of ranking by held-out source accuracy, AURCC is competitive when the labeled source is large and yields a more accurate ranking once it is small; the regime of interest in resource-constrained settings.