A Closer Look at Failure Modes in Temporal Understanding of Large Audio-Language Models

2026-06-15

AI Summary

This work addresses the limited performance of current large audio-language models on temporal reasoning tasks and the absence of effective benchmarks to elucidate their failure mechanisms. The authors construct a temporal understanding benchmark comprising 1,657 questions across three fundamental tasks and introduce causal mechanism analysis for the first time in this context. Their investigation reveals that redistributing attention is more effective than merely enhancing audio-specific attention, and that modality imbalance is not the sole bottleneck. Through behavioral analysis and intervention techniques—such as attention scaling at bottleneck layers—they improve accuracy from 55.9% to 59.1% without fine-tuning, demonstrating the efficacy of strategies that prioritize task-relevant audio tokens.

Abstract

Large Audio Language Models (LALMs) achieve strong performance on a variety of audio understanding tasks but continue to struggle with temporal reasoning, a fundamental capability central to human auditory perception. Understanding the causes of these failures remains challenging as existing benchmarks report performance gaps without probing underlying mechanisms. To address this, we introduce a benchmark with 1,657 questions across three foundational tasks designed specifically for mechanistic analysis. Examining model outputs across varying input settings (behavioral analysis) reveals that models often under-utilize audio when textual cues are available. We also provide the first causal mechanistic analysis of temporal reasoning failures in LALMs. Comparing attention upweighting against scaling, we find that redistributing attention across audio tokens is more effective than increasing audio attention. Targeting task-relevant tokens yields further gains. These findings suggest that modality imbalance alone cannot explain failures. Attention scaling at bottleneck layers improves accuracy from 55.9% to 59.1% without fine-tuning, demonstrating a promising direction for future work.

Problem

• temporal reasoning
• Large Audio Language Models
• failure modes
• audio understanding
• mechanistic analysis

Innovation

• temporal reasoning
• causal mechanistic analysis
• attention redistribution
• audio-language models
• modality imbalance