Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces MixProLAP, a probabilistic framework for audio-language pretraining that addresses the inherent ambiguity in audio-text alignment caused by overlapping sound events and diverse textual descriptions. By modeling each modality as a distribution rather than using deterministic embeddings, the approach captures uncertainty and improves cross-modal alignment through mixed audio-text pairs that simulate real-world acoustic environments. Experimental results demonstrate that MixProLAP significantly outperforms traditional deterministic methods in audio-text retrieval tasks, highlighting its effectiveness in handling many-to-many correspondence ambiguity.
Mixing audio-text pairs reveals that probabilistic modeling can dramatically enhance alignment accuracy in complex acoustic environments.
Acoustic environments often contain multiple overlapping sound events, and the same acoustic scene can be described using diverse textual expressions, making audio-text alignment inherently ambiguous. This paper proposes a probabilistic audio-language pretraining framework to model many-to-many correspondence ambiguity in audio-text alignment. Unlike conventional contrastive methods that learn deterministic point embeddings, our approach represents each modality as a distribution and learns uncertainty-aware cross-modal alignment. Rather than relying on masking-based uncertainty simulation, we mix audio-text pairs to create overlapping sounds that better reflect real acoustic mixtures and capture semantic inclusion relations among sound events. We further introduce a multi-level inclusion loss to enforce representations consistent with these relations. Experiments on audio-text retrieval benchmarks show that the proposed method outperforms deterministic baselines.