Listen, Reason, and Segment: Aligning LALMs with Editorial Judgment for Media Chapterization

2026-08-17

AI Summary

This study addresses the inability of Large Audio-Language Models (LALMs) to perform media chapterization due to insufficient editorial judgment. We propose AudioChaps, a framework that innovatively bypasses supervised fine-tuning cold-start by directly aligning models via Group Relative Policy Optimization (GRPO) integrated with Chain-of-Thought (CoT) reasoning, supported by a specialized dataset designed to enhance editorial decision-making. Experimental results demonstrate that AudioChaps-R1 achieves a 49-point F1-score improvement over state-of-the-art baselines, successfully enabling the precise conversion of unstructured audio into navigable structured media. These findings significantly advance the practical utility of LALMs in real-world production workflows by effectively bridging the gap between raw audio processing and structured content organization through reinforcement learning-based alignment.

Abstract

Large Audio Language Models (LALMs) have made rapid progress on standardized benchmarks, yet their deployment in practical media workflows, curation, archival indexing, and content distribution remains largely unrealized. We identify automated audio chapterization, the task of segmenting continuous audio streams into thematically coherent chapters, as a demanding and commercially consequential setting that exposes this gap. Chapterization is challenging because boundaries are defined less by objective acoustic events than by subjective editorial judgment, requiring models to reason sequentially over long acoustic contexts and approximate creator-authored boundary decisions. We present AudioChaps, a post-training framework for aligning end-to-end LALMs for this task via Group Relative Policy Optimization (GRPO) guided by Chain-of-Thought (CoT) reasoning. To support training and evaluation, we curate three datasets: AudioChaps-Alignment, derived from creator-annotated chapter boundaries on YouTube; AudioChaps-CoT, which provides structured supervision for well-formatted, high-quality, and evidence-grounded boundary reasoning; and AudioChaps-Eval, a held-out benchmark for audio chapterization. Applying GRPO directly without a Supervised Fine-Tuning (SFT) cold start, AudioChaps-R1-Zero already improves average F1 by 33 points over the state-of-the-art LALM Audio-Flamingo-3-Think. The AudioChaps framework produces our final aligned LALM, AudioChaps-R1, which improves average F1 by 49 points. These results demonstrate that GRPO-trained LALMs can reliably transform unstructured auditory streams into navigable, structured media. Our code, models, and dataset resources will be released upon acceptance at https://github.com/ta012/AudioChaps.

Problem

• Audio Chapterization
• Large Audio Language Models
• Editorial Judgment
• Media Workflows

Innovation

• Audio Chapterization
• Group Relative Policy Optimization
• Chain-of-Thought Reasoning
• Large Audio Language Models
• Post-training Alignment