Search papers, labs, and topics across Lattice.
This paper introduces PRISM, a topology-aware framework for cross-modal imputation in multimodal federated graph learning, addressing the challenge of client-level modality deficiency where clients lack certain modalities. By leveraging the federation to retrieve missing modality semantics and integrating them into local graph propagation, PRISM enhances the learning process for clients with incomplete representations. Experimental results demonstrate that PRISM outperforms existing state-of-the-art methods by an average of 4.48% across various multimodal graph datasets, highlighting its effectiveness in improving modality-deficient clients.
PRISM boosts performance in modality-deficient federated graph learning by intelligently retrieving and integrating missing modalities from the entire federation.
Multimodal federated graph learning (MM-FGL) aims to collaboratively learn from decentralized graphs with text and images. However, real-world clients may not share a common modality basis: a visual-search client may contain image--interaction graphs but no seller descriptions, while a catalog client may provide text but no product images. We refer to this practical setting as client-level modality deficiency. Unlike random instance-wise missingness, a deficient client lacks the local semantic basis needed to reconstruct the absent modality. More importantly, in graph learning, incomplete representations initialize message passing, so imputation errors can be filtered, mixed, and amplified by the receiving topology. To address this gap, we propose \textbf{PRISM} (\textbf{P}roactive \textbf{R}etrieval and \textbf{I}mputation via \textbf{S}tructural \textbf{M}eta-prompting), a topology-aware federated cross-modal imputation framework. Rather than reconstructing the missing modality solely from local observations, PRISM recovers missing-modality semantics from the federation and introduces them into local graph propagation under topology-aware control. Experiments on six multimodal graph datasets across graph-centric and modality-centric tasks show that PRISM consistently improves modality-deficient clients, outperforming state-of-the-art baselines by \textbf{4.48}\% on average.
