Track trials, findings, and clinical evidence as it lands.
Search papers, labs, and topics across Lattice.
This paper addresses the challenge of segmenting co-occurring white matter hyperintensities (WMH) and ischaemic stroke lesions (ISL) in MRI by developing hetero-modal learning strategies to maximize data utility from incomplete datasets. Hetero-modal learning significantly outperforms uni-modal approaches, particularly in low-data regimes with missing modalities, and enables inference on unseen modality combinations. The novel Multimodal Attention Router (MMAR) block, trained with corruption augmentation, further enhances robustness by dynamically weighting modality features to maintain performance even with unflagged corrupted inputs.
AI can now segment confounding brain lesions from corrupted, incomplete MRI scans by dynamically routing information across modalities.
White matter hyperintensities (WMH) and ischaemic stroke lesions (ISL) are visually confounding, co-occurring pathologies that require large, diverse datasets for robust deep learning segmentation. However, assembling such datasets is hindered by cohort samples that lack reference segmentations from both features and complete sets of MRI structural sequences (i.e.,"modalities"). To maximise data utility, we investigate hetero-modal learning using a dataset of 206 vascular disease patients across four MRI sequences (T1-weighted, T2-weighted, fluid-attenuated inversion recovery, and diffusion-weighted imaging) with expert annotations of both WMH and ISL. We demonstrate that hetero-modal learning outperforms models trained using a single imaging modality in scenarios with substantial missing data, including a split where only 10% of the training data contains all four modalities while the remainder is uni-modal, and a split relying on a single shared"anchor"modality with zero overlap between the remaining modalities. Furthermore, models trained under this second split successfully perform inference on unseen combinations of modalities. Yet, while standard hetero-modal networks handle missing sequences, clinical deployment introduces the additional challenge of silent data degradation - where modalities are present but severely corrupted. To bridge this gap, we introduce the Multimodal Attention Router (MMAR) block. Our experiments demonstrate that, when trained with a"corruption augmentation"strategy, the MMAR effectively dynamically weights the encoded features of each modality, maintaining strong performance during hetero-modal inference even in the presence of unflagged catastrophically corrupted modalities.