Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models

2026-04-15

AI Summary

This work addresses the susceptibility of unified audio-language models to temporal smoothing bias during generation, which hinders their effective utilization of transient acoustic cues and results in insufficient fine-grained alignment between output text and audio. To mitigate this issue, the authors propose a training-free temporal contrastive decoding method that, at inference time, constructs a contrastive signal between the original input and a temporally blurred “slow-path” view to dynamically refine the logits of the next token. The approach introduces, for the first time, a self-normalized stability score coupled with an uncertainty-aware gating mechanism, integrating waveform-smoothed recoding, adaptive blurring windows, and token-level logit updates to enable on-demand, precise enhancement of transient audio information. Evaluated on the MMAU and AIR-Bench benchmarks, the method consistently improves performance across multiple strong baseline models, demonstrating both effectiveness and architectural generality.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Large audio-language models (LALMs) generalize across speech, sound, and music, but unified decoders can exhibit a \emph{temporal smoothing bias}: transient acoustic cues may be underutilized in favor of temporally smooth context that is better supported by language priors, leading to less specific audio-grounded outputs. We propose \emph{Temporal Contrastive Decoding} (TCD), a training-free decoding method for unified LALMs that mitigates this effect at inference time. TCD constructs a temporally blurred slow-path view by smoothing the input waveform and re-encoding it, then contrasts next-token logits from the original and slow-path views. The contrastive signal is applied as a token-level logit update restricted to a small candidate set. A self-normalized stability score sets the blur window and update scale, and a step-wise gate based on uncertainty and audio reliance activates the update only when needed. Experiments on MMAU and AIR-Bench show consistent improvements on strong unified LALMs. We further conduct ablations and an architectural applicability study to analyze the contributions of key components and how TCD behaves across large audio-language model designs.

Problem

• temporal smoothing bias
• large audio-language models
• transient acoustic cues
• audio-grounded generation

Innovation

• Temporal Contrastive Decoding
• training-free decoding
• audio-language models
• temporal smoothing bias
• inference-time adaptation