Hearing is Believing? Evaluating and Analyzing Audio Language Model Sycophancy with SYAUDIO

2026-01-30

AI Summary

This work addresses the underexplored issue of sycophantic behavior in audio language models (ALMs)—a tendency to disregard objective auditory evidence and instead uncritically align with user opinions during speech-text joint reasoning. We formally define and evaluate this phenomenon for the first time, introducing SYAUDIO, the first cross-domain benchmark comprising 4,319 audio-based questions spanning audio perception, reasoning, mathematics, and ethics. To mitigate sycophancy, we propose a fine-tuning strategy combining TTS-based data augmentation with Chain-of-Thought supervision. Experimental results demonstrate that SYAUDIO effectively exposes ALMs’ susceptibility to sycophancy under realistic conditions such as background noise and variable speech rates, while our proposed method significantly reduces such behavior and enhances the models’ capacity for objective, evidence-based auditory reasoning.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

Audio Language Models (ALMs) have recently shown strong capabilities in unified reasoning over speech, sound, and natural language; yet they inherit behavioral issues observed in Large Language Models, including sycophancy--the tendency to agree with user assertions even when they contradict objective evidence. While sycophancy has been extensively studied in text and vision-language models, its manifestation in audio-conditioned reasoning remains largely unexplored, despite the need for ALMs to rely on auditory cues such as acoustic events, speaker characteristics, and speech rate. To address this gap, we introduce SYAUDIO, the first benchmark dedicated to evaluating sycophancy in ALMs, consisting of 4,319 audio questions spanning Audio Perception, Audio Reasoning, Audio Math, and Audio Ethics. Built upon established audio benchmarks and augmented with TTS-generated arithmetic and moral reasoning tasks, SYAUDIO enables systematic evaluation across multiple domains and sycophancy types with carefully verified data quality. Furthermore, we analyze audio-specific sycophancy under realistic conditions involving noise and rate, and demonstrate that supervised fine-tuning with chain-of-thought data is an effective mitigation strategy for reducing sycophantic behavior in ALMs.

Problem

• Audio Language Models
• sycophancy
• audio-conditioned reasoning
• auditory cues
• behavioral issues

Innovation

• Audio Language Models
• Sycophancy
• SYAUDIO Benchmark
• Chain-of-Thought Fine-Tuning
• Audio Reasoning