Search papers, labs, and topics across Lattice.
This paper introduces FedCoRe, a novel approach for addressing missing modalities in federated learning (FL) settings within healthcare, specifically targeting respiratory deterioration tasks. By leveraging paired supervision from clients that have both complete and incomplete modality data, FedCoRe learns to correct predictions in representation space rather than generating synthetic data. The results show that this method can recover up to 52.8% of lost performance when modalities like ECG or CXR are missing, highlighting its effectiveness in real-world deployment scenarios where data availability varies across hospitals.
FedCoRe can recover nearly half of the performance lost due to missing ECG or CXR data in federated healthcare models, transforming how we handle incomplete patient information.
Federated multimodal models often assume every site has every modality, although hospitals differ in access to EHRs, chest radiographs, and ECGs. We study this setting on a MIMIC-derived respiratory deterioration task with simulated FL clients and introduce FedCoRe (Federated Cross-Modal Representation Completion). FedCoRe learns representation- or logit-space corrections rather than generating synthetic ECGs or CXR images. When a client observes a modality that may be missing at deployment, it evaluates the same example with and without that modality to obtain paired supervision. Only clients with such pairs update the completion module, and validation may retain the unchanged prediction. We freeze the trained multimodal predictor during evaluation so that measured differences come only from completion. Hiding ECG reduced AUROC by about 0.085; paired-example FedAvg restored 0.0415 AUROC, or 49.0% of the lost performance. We therefore report two distinct effects: paired-example FedAvg partially recovers the missing-ECG gap, while validation-selected completion is a task-specific classifier-logit correction rather than literal ECG recovery. For CXR, effect-aware completion recovers 52.8% of the loss in a controlled test where CXR is hidden. Paired-example FedAvg transfers part of this effect, but validation keeps the no-completion baseline for deployment cases whose inputs lack CXR. Thus, FedCoRe should be read as a validation-gated completion/correction framework: it can recover missing-modality signal in supported settings, but it should be deployed only when paired examples and validation evidence support that modality.
NVIDIA