Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Challenging the standard practice of relying exclusively on final-layer vision-language model (VLM) embeddings for zero-shot out-of-distribution (OOD) detection, this work shows that intermediate representations capture critical, modality-dependent OOD signals in medical imaging. The authors reveal that prior layer-selection heuristics based on single-resolution entropy minimization are dangerously brittle, inducing up to 19.3% AUROC variance based solely on arbitrary binning choices. To resolve this, they introduce a multi-resolution entropy estimation framework that stabilizes layer combination by aggregating histogram statistics across multiple discretization scales, consistently outperforming state-of-the-art zero-shot baselines across the MIDOG and OASIS benchmarks.
Final VLM layers discard crucial safety signals in medical domains, but naive entropy-based intermediate layer selection fluctuates by up to 19.3% AUROC based on arbitrary binning—a failure mode eliminated by multi-scale discretization.
Out-of-distribution (OOD) detection is crucial for safe deployment of medical AI systems, where domain shifts arise across institutions, acquisition protocols, and patient populations. VLMs enable zero-shot OOD detection by embedding images into a language-aligned latent space, where cross-modal similarity serves as a non-parametric confidence signal for identifying in-distribution samples. Yet existing methods rely almost exclusively on final-layer embeddings, implicitly assuming that the deepest representations are universally optimal. We first show that this assumption does not hold in medical imaging: intermediate layers provide complementary OOD signals, and the optimal representational depth depends on the respective image modality. While prior work selects layer combinations via entropy minimization of normalized histograms, we demonstrate that single-resolution entropy estimation is highly sensitive to binning choices, leading to performance variations of up to 19.3% AUROC. To address this instability, we propose a multi-resolution entropy estimation strategy that aggregates histogram statistics across multiple discretization scales, enabling robust and stable intermediate-layer selection. Across two medical OOD benchmarks, namely MIDOG and OASIS, covering distinct imaging modalities, diverse shift types, and different VLM backbones, our method consistently outperforms state-of-the-art approaches, offering a lightweight and stable solution for zero-shot OOD detection.