Pushing the Limits of End-to-End Diarization

2025-09-18

AI Summary

This study addresses the limited accuracy of joint segmentation and clustering in end-to-end speaker diarization. We propose a unified solution based on an extended non-autoregressive EEND-TA architecture. To enhance modeling capability for complex overlapping speech and long-tail scenarios, we construct a large-scale synthetic dataset covering up to eight simultaneous speakers and adopt a pretraining-finetuning paradigm to improve generalization. Our method directly outputs speaker label sequences without requiring post-processing steps such as clustering or voice activity detection (VAD), thereby simplifying the pipeline and improving robustness. Evaluated on standard benchmarks—including AliMeeting, AMI, and DIHARD III—our approach achieves state-of-the-art performance, with a DER of 14.49% on DIHARD III, demonstrating strong cross-domain adaptability and practical deployability.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

In this paper, we present state-of-the-art diarization error rates (DERs) on multiple publicly available datasets, including AliMeeting-far, AliMeeting-near, AMI-Mix, AMI-SDM, DIHARD III, and MagicData RAMC. Leveraging EEND-TA, a single unified non-autoregressive model for end-to-end speaker diarization, we achieve new benchmark results, most notably a DER of 14.49% on DIHARD III. Our approach scales pretraining through 8-speaker simulation mixtures, ensuring each generated speaker mixture configuration is sufficiently represented. These experiments highlight that EEND-based architectures possess a greater capacity for learning than previously explored, surpassing many existing diarization solutions while maintaining efficient speeds during inference.

Problem

• Achieving state-of-the-art diarization error rates
• Scaling pretraining through multi-speaker simulation mixtures
• Enhancing EEND architecture learning capacity efficiently

Innovation

• EEND-TA non-autoregressive model for diarization
• Pretraining with 8-speaker simulation mixtures
• Achieves state-of-the-art diarization error rates