Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces H\"older++, a novel multimodal variational autoencoder that enhances the trade-off between generative quality and coherence by implementing H\"older pooling without approximation and utilizing an extended architecture that distinguishes between shared and private representations. The proposed method also incorporates hierarchical inference to improve the disentanglement of these representations. Experimental results demonstrate that H\"older++ not only enhances the generative quality-coherence trade-off but also produces more structured latent spaces and informative shared representations for downstream tasks.
H\"older++ achieves a breakthrough in multimodal VAEs by enhancing generative quality and coherence without sacrificing diversity.
Existing approaches for multimodal variational autoencoders (VAEs) face a trade-off between generative quality and coherence-i.e., they struggle to generate realistic and diverse samples that, at the same time, are semantically consistent across modalities. A recent work shows that using a simple approximation to H\"older pooling as an aggregation method improves coherence over the SOTA MMVAE+, despite assuming a single shared representation across all modalities. Yet, it slightly compromises sample diversity. Inspired by this insight, we propose H\"older++, a novel multimodal VAE that improves the generative quality-coherence trade-off through: (i) the first implementation of H\"older pooling without any approximation for multimodal VAEs; (ii) an extended architecture that models distinct shared and private (i.e., modality-specific) representations (H\"older+); and (iii) hierarchical inference that further enhances the disentanglement between the shared and private representations (H\"older++). Our experiments corroborate that H\"older++ consistently improves the generative quality-coherence trade-off, yields more structured latent spaces, and learns shared representations that are informative for downstream tasks.