MixProLAP: Mixture-Induced Uncertainty Modeling for Probabilistic Language-Audio Pretraining

2026-06-18

AI Summary

This work addresses the inherent many-to-many ambiguity in audio–text alignment, which arises from the superposition of multiple sound events in acoustic scenes and the diversity of textual descriptions. To tackle this challenge, the authors propose a probabilistic language–audio pretraining framework that represents each modality as a distribution rather than a deterministic embedding. By simulating realistic acoustic mixtures through mixed audio–text pairs, the model achieves uncertainty-aware cross-modal alignment. The approach innovatively incorporates mixture-based sound event modeling to capture semantic inclusion relationships and introduces a multi-level inclusion loss function, thereby overcoming the limitations of conventional contrastive learning paradigms that rely on deterministic representations. Experimental results demonstrate that the proposed method significantly outperforms existing deterministic baselines on standard audio–text retrieval benchmarks.

Abstract

Acoustic environments often contain multiple overlapping sound events, and the same acoustic scene can be described using diverse textual expressions, making audio-text alignment inherently ambiguous. This paper proposes a probabilistic audio-language pretraining framework to model many-to-many correspondence ambiguity in audio-text alignment. Unlike conventional contrastive methods that learn deterministic point embeddings, our approach represents each modality as a distribution and learns uncertainty-aware cross-modal alignment. Rather than relying on masking-based uncertainty simulation, we mix audio-text pairs to create overlapping sounds that better reflect real acoustic mixtures and capture semantic inclusion relations among sound events. We further introduce a multi-level inclusion loss to enforce representations consistent with these relations. Experiments on audio-text retrieval benchmarks show that the proposed method outperforms deterministic baselines.

Problem

• audio-text alignment
• ambiguity
• probabilistic modeling
• sound event overlap
• cross-modal correspondence

Innovation

• probabilistic pretraining
• uncertainty modeling
• audio-text alignment
• mixture-induced ambiguity
• multi-level inclusion loss