AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models

2026-06-12

AI Summary

This study addresses the severe redundancy in existing audio-language datasets, which hinders the post-training effectiveness of large models on complex audio reasoning tasks. To mitigate this issue, the authors propose a deduplication-enhanced data construction approach that, for the first time, integrates acoustic similarity–based deduplication to improve data diversity, standardizes multi-choice question formats, and leverages Qwen3-30B to generate chain-of-thought (CoT) supervision signals for building a high-quality, reasoning-oriented post-training dataset. Post-training Qwen2-Audio-7B-Instruct on this curated dataset yields significant performance gains across multiple audio reasoning benchmarks, including MMAU-mini, MMSU, and MMAR.

Abstract

Large Audio-Language Models (LALMs) have shown strong performance on a wide range of audio understanding tasks, yet they still struggle with complex audio reasoning. A practical way to improve such capabilities is post-training, whose effectiveness critically depends on the quality and diversity of training data. However, existing audio-language datasets often contain substantial redundancy, where many samples are highly similar in acoustic content and thus provide overlapping supervisory signals. Such redundancy not only increases annotation cost, but also limits corpus diversity and reduces the effectiveness of post-training. To address this issue, we propose a redundancy-aware data construction pipeline for building reasoning-oriented supervision for LALMs. Specifically, we first perform acoustic similarity-based deduplication across raw audio datasets to improve corpus diversity. We then integrate existing audio captions and question-answer pairs into a unified multiple-choice format. Based on these unified annotations, we leverage Qwen3-30B to generate chain-of-thought (CoT) rationales for reasoning-oriented supervision. Based on this pipeline, we construct AudioDER, a reasoning-oriented post-training dataset containing approximately 191k samples spanning sound, speech, and music. Each sample consists of an audio clip, a multiple-choice question, four answer candidates, an audio caption, and a CoT rationale. Extensive experiments show that post-training on AudioDER consistently improves the performance of Qwen2-Audio-7B-Instruct on multiple audio reasoning benchmarks, including MMAU-mini, MMSU, and MMAR. We hope AudioDER can serve as a valuable resource for advancing audio reasoning research and the development of more capable LALMs.

Problem

• audio-language models
• data redundancy
• post-training
• audio reasoning
• dataset deduplication

Innovation

• audio-language models
• deduplication
• chain-of-thought reasoning
• post-training dataset
• acoustic similarity