The Sound of Absence: Audio-Language Embedding Models Struggle with Negation

2026-07-13

AI Summary

This work addresses the limited ability of existing audio–language embedding models to comprehend negation, particularly their failure to distinguish between the presence and absence of sound events. To this end, we introduce NegEval-Audio, the first evaluation framework specifically designed for assessing negation understanding in audio–language models. We reformulate the AudioCaps and Clotho datasets into two tasks—Retrieval-Neg and MCQ-Neg—to systematically evaluate model performance in negated contexts. Our experiments reveal that state-of-the-art models, including CLAP, perform significantly below random chance on the MCQ-Neg task, uncovering a pervasive affirmative bias in current audio–language representations. These findings highlight a critical deficiency in how existing embedding spaces encode negation semantics, underscoring the need for more nuanced modeling of linguistic negation in multimodal audio–language systems.

Abstract

Audio-language embedding models such as CLAP are widely evaluated on matching present sound events, but rarely on negation. We show this affirmation-only evaluation hides a key limitation: these models fail to encode negated sound concepts, mapping affirmative and negated captions to nearly identical representations. To expose this blind spot, we introduce NegEval-Audio, a framework that converts existing datasets into two negation-aware tasks, Retrieval-Neg and Multiple-Choice Negation (MCQ-Neg), to probe whether models distinguish present from absent events. On AudioCaps and Clotho, performance degrades sharply under negation, with negation-type MCQ accuracy falling far below chance, and the failure persists even for a recent multimodal LLM-based embedding model. While a training-free steering method improves MCQ-Neg, it yields marginal gains for Retrieval-Neg. This indicates that affirmation bias is a fundamental flaw in the representation geometry, necessitating explicit negation-aware training objectives.

Problem

• audio-language embedding
• negation
• representation bias
• sound event absence
• affirmation bias

Innovation

• negation understanding
• audio-language embedding
• affirmation bias
• NegEval-Audio
• representation geometry