Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces General Incomplete Multimodal Learning (GIML), a novel framework that addresses both inter-modality missing and intra-modality degradation in multimodal learning. By modeling missing patterns as continuous information degradation, GIML employs a Noise-aware Quality Estimator to assess corruption levels and a Noise-Semantic Decoupled module to enhance robustness against noise interference. Extensive experiments show that GIML significantly improves performance across various datasets, highlighting its effectiveness in real-world applications where data corruption is common.
GIML achieves robust multimodal learning by effectively managing both missing modalities and corrupted features, outperforming existing methods in real-world scenarios.
Multimodal learning robust to missing modalities is essential for real-world applications. Existing methods mainly focus on inter-modality missing, where entire modalities are absent, while overlooking intra-modality degradation, where modalities are present but severely corrupted. In practice, these two types of missing often coexist, making existing approaches ineffective. To address this limitation, we propose General Incomplete Multimodal Learning (GIML), a unified framework that simultaneously handles both inter-modality missing and intra-modality degradation through dynamic quality perception. Specifically, GIML models heterogeneous missing patterns as continuous modality information degradation, enabling degradation-aware adaptive fusion. To achieve reliable quality perception, we introduce a Noise-aware Quality Estimator that learns the mapping from corrupted features to noise intensity through controlled noise injection. Furthermore, we propose a Noise-Semantic Decoupled module that separates semantic information from noise interference. This improves robustness and generalization to unseen corruption patterns. Extensive experiments across datasets with diverse modality types demonstrate the effectiveness and generality of GIML. Code is available at: https://github.com/Yu-Five/GIML.