Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To resolve how cross-modal interactions drive predictions, this work establishes a structured latent variable framework that factorizes multimodal representations into shared task-relevant, modality-specific predictive, and task-irrelevant components. Disentangling continuous representations via likelihood-based guidance provides a principled mechanism to isolate whether performance stems from individual signals, redundant features, or emergent cross-modal synergies. Across diverse multimodal benchmarks, this structured decomposition achieves robust empirical improvements over standard fusion baselines while enabling rigorous attribution of continuous multimodal interactions.
Multimodal models no longer need to rely on entangled black-box fusion: invertible flows and low-rank latent priors can explicitly isolate shared task signals from emergent cross-modal synergy.
Multimodal prediction relies on diverse forms of evidence: information repeated across modalities, cues specific to a single source, and complex cross-modal dependencies that emerge only when inputs are considered together. While recent methods promote richer interactions, they lack a principled way to isolate these target-relative contributions within learned continuous representations. We introduce a framework that applies contrastive or masked objectives at intermediate layers, coupled with source-wise invertible normalizing flows and a supervised, low-rank latent variable model. This architecture explicitly factorizes the joint distribution into shared task-relevant variation, modality-specific predictive variation, and task-irrelevant dependence. Drawing connections to prior multimodal learning assumptions, our approach evaluates how modalities independently and jointly contribute to the target. Ultimately, this framework unites intermediate representation learning with structured likelihood-based guidance, offering a practical latent-variable lens for characterizing continuous multimodal interactions. Empirically, we demonstrate the effectiveness of our approach across diverse multimodal benchmarks, showing robust improvements in predictive performance.