Search papers, labs, and topics across Lattice.
ProtoCLIP refines CLIP-style VLMs for chest X-ray classification by addressing label co-occurrence, class imbalance, and domain shift via targeted data curation and distilled anchor alignment. It constructs pathology-focused training subsets with curated negative samples and introduces a representation-preserving distillation objective to stabilize adaptation. Evaluated on VinDr-CXR, ProtoCLIP improves AUC by 2-10 percentage points over CLIP, achieving a state-of-the-art AUC of 0.94 for pneumothorax.
Achieve state-of-the-art zero-shot chest X-ray classification by cleverly curating training data and distilling knowledge into CLIP models, sidestepping the need for large-scale retraining.
Zero-shot vision-language models (VLMs) have shown promise for chest radiograph classification, but their performance is often limited by confounding label co-occurrence, long-tail class imbalance, and transfer instability under domain shift. We propose ProtoCLIP, a refinement strategy for CLIP-style VLMs that improves zero-shot discrimination through targeted data curation and distilled anchor alignment. Specifically, we construct pathology-focused training subsets with curated negative samples to reduce co-occurrence bias. We also introduce a representation-preserving distillation objective to stabilize adaptation while maintaining semantic structure and improving discrimination of clinically relevant co-occurring pathologies. Evaluated on an unseen dataset VinDr-CXR, ProtoCLIP improves AUC by 2-10 percentage points over a strong CLIP-based baseline across multiple findings. For pneumothorax specifically, ProtoCLIP achieves a state-of-the-art AUC of 0.94. These results demonstrate that anchor-guided refinement, coupled with curated supervision and controlled adaptation, can mitigate common zero-shot transfer failures in medical VLMs without requiring large-scale retraining.
