Who Wins the Conflict? Mechanistic Interpretability of Text Bias in Audio LLMs

2026-06-17

AI Summary

This work addresses the susceptibility of audio large language models to textual dominance bias under audio-text conflict, which often leads to hallucinations. Through mechanistic interpretability analysis, we uncover—for the first time—a functional pathway separation between text and audio within the model, followed by late-stage fusion in a shared semantic space. Crucially, we reveal that audio information is actively suppressed rather than erased during processing. Building on this insight, we propose back-patching, a training-free intervention that retroactively propagates late-layer audio activations back to early layers to strengthen audio representations. Experimental results demonstrate that this approach substantially mitigates textual dominance bias and significantly enhances multimodal alignment in conflicting scenarios.

Abstract

While Audio Large Language Models (Audio LLMs) excel at multimodal understanding, they suffer from text dominance, a bias where models blindly favor text over acoustic evidence, causing hallucinations. However, the internal mechanisms underlying how these models behave when audio and textual inputs contradict each other remain unexplored. In this work, we present the first mechanistic analysis of this phenomenon by tracing the propagation of internal representations across layers. Our investigation reveals three key findings: (i) text dominance is systematically and empirically across models; (ii) while text and audio rely on functionally distinct pathways, they ultimately converge into a shared semantic space in late layers; and (iii) the text pathway does not erase audio information, but rather actively suppresses intact audio representations. Building on these insights, we leverage back-patching, a training-free intervention that routes late-layer audio activations back into earlier layers. This amplifies the audio representations, enabling them to overcome textual suppression. Our evaluation shows that back-patching consistently reduces text dominance, paving the way for mechanistic multimodal alignment under conflict.

Problem

• text dominance
• audio LLMs
• multimodal conflict
• hallucination
• mechanistic interpretability

Innovation

• mechanistic interpretability
• text dominance
• audio LLMs
• back-patching
• multimodal alignment