Search papers, labs, and topics across Lattice.
The paper introduces COrAL, a novel framework for multimodal contrastive learning that explicitly models redundant, unique, and synergistic information by employing a dual-path architecture with orthogonality constraints to disentangle shared and modality-specific features. To promote synergy modeling, COrAL uses asymmetric masking with complementary view-specific patterns, forcing the model to infer cross-modal dependencies. Experiments on synthetic and MultiBench datasets demonstrate that COrAL achieves state-of-the-art performance with low variance, indicating more stable and comprehensive multimodal embeddings.
By explicitly disentangling redundant, unique, and synergistic information, COrAL achieves more stable and comprehensive multimodal embeddings compared to existing contrastive learning methods.
Multimodal learning seeks to integrate information from heterogeneous sources, where signals may be shared across modalities, specific to individual modalities, or emerge only through their interaction. While self-supervised multimodal contrastive learning has achieved remarkable progress, most existing methods predominantly capture redundant cross-modal signals, often neglecting modality-specific (unique) and interaction-driven (synergistic) information. Recent extensions broaden this perspective, yet they either fail to explicitly model synergistic interactions or learn different information components in an entangled manner, leading to incomplete representations and potential information leakage. We introduce \textbf{COrAL}, a principled framework that explicitly and simultaneously preserves redundant, unique, and synergistic information within multimodal representations. COrAL employs a dual-path architecture with orthogonality constraints to disentangle shared and modality-specific features, ensuring a clean separation of information components. To promote synergy modeling, we introduce asymmetric masking with complementary view-specific patterns, compelling the model to infer cross-modal dependencies rather than rely solely on redundant cues. Extensive experiments on synthetic benchmarks and diverse MultiBench datasets demonstrate that COrAL consistently matches or outperforms state-of-the-art methods while exhibiting low performance variance across runs. These results indicate that explicitly modeling the full spectrum of multimodal information yields more stable, reliable, and comprehensive embeddings.
