Search papers, labs, and topics across Lattice.
This study investigates the presence of real-world shortcuts in the MedCLIP model, a vision-language model for medical imaging, by attaching 17 linear classification probes to its ResNet-50 layers. The authors find that while the final probes achieve high AUROC scores, they exhibit poor calibration, indicating that the model is susceptible to shortcuts that manifest at various depths鈥攍ocalized shortcuts appear in later layers, while diffuse shortcuts are evident in earlier layers. The analysis highlights critical data quality issues in the datasets used, emphasizing the necessity for better-annotated data to ensure robust model performance in medical AI applications.
Even state-of-the-art medical AI models like MedCLIP are vulnerable to real-world shortcuts that can compromise diagnostic reliability.
Vision-language models, such as contrastive language-image pre-training (CLIP)-based approaches, have reached state-of-the-art (SOTA) results in medical artificial intelligence. However, recent work reveals that CLIP-based models remain vulnerable to shortcuts. We investigate how real-world shortcuts manifest across different layers of the medical CLIP-based model, MedCLIP, and its vision encoder, a frozen ResNet-50. We attach 17 linear classification probes to the intermediate layers of the ResNet-50 and train them on three different dataset configurations and targets: NIH-CXR14 (pneumothorax) and PadChest (cardiomegaly and pneumothorax). This setup allows us to observe model behaviour during evaluation using subgroup-based calibration and layer-wise confidence curves. We find that the final linear probes achieve a high AUROC but poor calibration in the models. The layer-wise confidence analyses suggest that shortcuts emerge at different depths. Patterns consistent with localised shortcuts, such as drains, appear at later layers, while patterns consistent with diffuse shortcuts, such as scanner-specific noise patterns, emerge earlier, aligning with previous work. Finally, we conduct a manual analysis of the images, which reveals data quality issues in both NIH-CXR14 and PadChest. Our findings underscore that even SOTA models remain vulnerable to shortcuts, and the need for high-quality and well-annotated datasets to draw solid conclusions. Code can be found on our GitHub: https://github.com/nikodice4/MedCLIP_shortcuts.
