Reasoning LLM Improves Speaker Recognition in Long-form TV Dramas

2026-07-02

AI Summary

This work addresses the challenge of speaker recognition in short audio clips from long-form TV dramas, where weak acoustic features often hinder performance. To tackle this issue, the authors propose DramaSR-LRM, a novel approach that introduces large reasoning models (LRMs) to the task for the first time. The method dynamically aggregates contextual evidence by autonomously invoking multimodal tools to fuse auditory, linguistic, and visual cues. Alongside the proposed method, the authors construct and release DramaSR-532K, a large-scale multimodal benchmark dataset comprising 532,000 character-annotated dialogue segments. Experimental results demonstrate that DramaSR-LRM substantially outperforms existing approaches, particularly excelling in scenarios with limited acoustic information.

Abstract

Long-form TV dramas present a formidable challenge for comprehensive video understanding, where deciphering complex storyline often relies on \textbf{speaker recognition}, the task of accurately attributing each spoken utterance to its respective character. In this paper, we advance this field through two primary contributions. (1) We introduce \textbf{DramaSR-532K}, a large-scale benchmark comprising 532K annotated dialogue lines across more than 900 unique characters, necessitating the integration of auditory, linguistic, and visual cues for speaker recognition. (2) We propose \textbf{DramaSR-LRM}, a robust approach built upon a large reasoning model (LRM). DramaSR-LRM is designed to autonomously aggregate contextual evidence via multimodal tool-use, synthesizing diverse inputs to achieve high-fidelity attribution. Experimental results demonstrate that DramaSR-LRM significantly outperforms existing baselines, particularly on short utterances where acoustic biometrics are inherently unreliable. \textit{All the data and code will be made publicly available at the project page: https://www.github.com/198808xc/DramaSR-LRM.}

Problem

• speaker recognition
• long-form TV dramas
• video understanding
• multimodal reasoning
• dialogue attribution

Innovation

• speaker recognition
• large reasoning model
• multimodal tool-use
• long-form TV dramas
• DramaSR-532K