Search papers, labs, and topics across Lattice.
This paper introduces DistMedVL, a probabilistic framework for vision-language alignment in medical image segmentation that explicitly addresses uncertainty in both visual and textual modalities. By employing a lightweight Probabilistic Cross-Modal Adapter (PCM-Adapter) that integrates a Mahalanobis Alignment Module and a Distribution Flow Module, the method effectively models representational uncertainty and enhances compatibility between image patches and textual tokens. Experimental results across eight medical segmentation benchmarks reveal that DistMedVL significantly outperforms existing methods while maintaining a low parameter count, demonstrating improved data efficiency and robustness to perturbations and domain shifts.
Uncertainty-aware segmentation can drastically improve performance in medical imaging, with DistMedVL achieving superior results using only 6.3M parameters.
Cross-modal alignment of visual and textual representations is fundamental to multimodal medical image understanding, yet remains hindered by uncertainty in both modalities under real-world clinical conditions. Existing vision-language segmentation methods rely on deterministic cross-modal matching, which overlooks aleatoric uncertainty from ambiguous boundaries and epistemic uncertainty from limited training data, leading to fragile performance under domain shift. To address this issue, we propose DistMedVL, a probabilistic vision-language framework that introduces a lightweight Probabilistic Cross-Modal Adapter (PCM-Adapter) upon frozen encoders to explicitly model representational uncertainty. Specifically, the PCM-Adapter comprises two sequential modules for progressive probabilistic alignment. We first devise a Mahalanobis Alignment Module (MAM) that models textual tokens as Gaussian distributions and computes patch-text compatibility via Mahalanobis distance, yielding variance-conditioned matching that downweights unreliable feature dimensions. Moreover, we devise a Distribution Flow Module (DFM) that estimates modality-wise confidence parameters and performs vision-guided refinement of textual distributions, accommodating distributional variation across imaging modalities. Extensive experiments across eight medical segmentation benchmarks demonstrate that DistMedVL outperforms state-of-the-art methods with only 6.3M trainable parameters, exhibiting superior data efficiency, perturbation robustness and cross-dataset generalization.
