Search papers, labs, and topics across Lattice.
This paper introduces CXR-Retrieve, a benchmark designed for compositional text-to-image retrieval specifically in chest radiography, addressing the limitations of existing models that focus on report-to-image matching rather than clinical query satisfaction. By incorporating a label-aware contrastive fine-tuning objective, the authors enhance the retrieval process to ensure that images meet all asserted pathology constraints, including conjunctions and negations. The results demonstrate significant improvements in retrieval precision, with an 8.5 percentage point increase for conjunctions and a 22.0 percentage point increase for negation queries, highlighting the importance of accurately modeling clinical assertions in retrieval tasks.
CXR-Retrieve reveals that effective chest X-ray retrieval hinges on understanding not just what findings are present, but how they are clinically asserted, leading to substantial gains in precision.
Large chest radiography archives are difficult to search because most studies are paired only with free-text reports rather than structured clinical annotations. Vision-language models offer a natural interface for text-to-image retrieval, but current biomedical models are primarily optimized for report-to-image matching rather than for satisfying short clinical search queries. This creates an objective mismatch: a model may retrieve images related to words in the query while failing to satisfy the full clinical constraint, especially for conjunctions and negations such as ``atelectasis and no pneumonia.''We introduce CXR-Retrieve, a structured benchmark for compositional chest X-ray text-to-image retrieval. The benchmark contains 5,159 test images from the official test-split of MIMIC-CXR-JPG and 145 textual queries spanning single and conjunction findings, both positive and negative. Relevance is defined by whether a retrieved image satisfies all asserted pathology constraints, rather than by whether it matches a paired report. We further propose a label-aware contrastive fine-tuning objective for clinical retrieval. Our method attracts image-text pairs with compatible asserted pathology constraints, including shared confirmed absences, while explicitly repelling contradictory pairs. Starting from the in-domain CXR-CLIP checkpoint, our method improves Precision@5 over CXR-CLIP by 8.5 percentage points on two-pathology conjunctions and by 22.0 percentage points on negation queries. These results show that reliable chest X-ray retrieval requires training objectives that model not only which findings are mentioned, but also how they are clinically asserted.
