Search papers, labs, and topics across Lattice.
This paper introduces MOSAIC, a novel modality-agnostic federated learning framework designed for weakly supervised binary tumor segmentation in clinical settings with client-specific missing modalities. By employing a client-specific modality-alignment module and a spectral prototype alignment loss, MOSAIC effectively reconciles distribution shifts across institutions, achieving significant improvements in segmentation accuracy without requiring complete modality data. Experiments on multiple brain tumor benchmarks reveal that MOSAIC approaches fully supervised performance using only image-level labels, with a notable 0.84 Dice score on the FeTS2022 dataset.
Achieving near fully supervised accuracy in tumor segmentation using only image-level labels, even in the presence of client-specific missing modalities, is a game changer for federated learning in healthcare.
Trustworthy multimodal fusion in clinical settings requires handling incomplete and heterogeneous modality subsets across institutions, where privacy constraints prohibit centralized data sharing. Federated learning (FL) mitigates data-sharing constraints but suffers from client-specific missing modalities, where institutions possess incomplete multimodal subsets, degrading fusion quality and segmentation performance. While FL and weak supervision have been studied separately, their joint use with image-level labels under heterogeneous missing modalities remains unaddressed. We propose \textbf{MOSAIC}, the first modality-agnostic federated framework for weakly supervised binary tumor segmentation under client-specific missing modalities. We introduce a client-specific modality-alignment module that fuses available channels into a shared latent space without prior knowledge of modality identity, a spectral prototype alignment loss that reconciles cross-client distribution shift using compact non-invertible frequency-domain statistics, and a dedicated federated refinement network that denoises the resulting CAM pseudo-labels into accurate masks, breaking the accuracy ceiling of weak supervision. Experiments on three multi-institutional brain tumor benchmarks (FeTS2022, BraTS-MEN, and BraTS-SSA) demonstrate significant improvements over all image, box, and point-supervised baselines, approaching fully supervised accuracy using only image-level labels and reaching 0.84 Dice on FeTS2022. Dynamic new client addition enables previously unseen institutions to join an already-trained federation within 0.01-0.04 Dice without retraining. Code is available at https://github.com/Tarun2201/MOSAIC.
