Recent Trends in Distant Conversational Speech Recognition: A Review of CHiME-7 and 8 DASR Challenges

2025-07-24

AI Summary

This work addresses the joint automatic speech recognition (ASR) and speaker diarization task for far-field, multi-channel conversational speech under complex acoustic conditions. Methodologically, we propose an end-to-end, generalizable joint modeling framework integrating pretrained end-to-end ASR, neural speech separation enhancement, guided source separation, and speaker diarization refinement, augmented by multi-system fusion for robustness. A key contribution is the empirical validation of large language models’ strong tolerance to downstream transcription errors, highlighting the effectiveness of target-speaker diarization optimization and separation-guided strategies. Evaluated on benchmarks including CHiME-8, our 32 submitted systems consistently demonstrate the dominance of end-to-end architectures. However, performance degradation persists in high-noise and cross-device scenarios, indicating room for further improvement.

Technology Category

• Natural Language Processing: Conversational AI/Dialog Systems
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The CHiME-7 and 8 distant speech recognition (DASR) challenges focus on multi-channel, generalizable, joint automatic speech recognition (ASR) and diarization of conversational speech. With participation from 9 teams submitting 32 diverse systems, these challenges have contributed to state-of-the-art research in the field. This paper outlines the challenges' design, evaluation metrics, datasets, and baseline systems while analyzing key trends from participant submissions. From this analysis it emerges that: 1) Most participants use end-to-end (e2e) ASR systems, whereas hybrid systems were prevalent in previous CHiME challenges. This transition is mainly due to the availability of robust large-scale pre-trained models, which lowers the data burden for e2e-ASR. 2) Despite recent advances in neural speech separation and enhancement (SSE), all teams still heavily rely on guided source separation, suggesting that current neural SSE techniques are still unable to reliably deal with complex scenarios and different recording setups. 3) All best systems employ diarization refinement via target-speaker diarization techniques. Accurate speaker counting in the first diarization pass is thus crucial to avoid compounding errors and CHiME-8 DASR participants especially focused on this part. 4) Downstream evaluation via meeting summarization can correlate weakly with transcription quality due to the remarkable effectiveness of large-language models in handling errors. On the NOTSOFAR-1 scenario, even systems with over 50% time-constrained minimum permutation WER can perform roughly on par with the most effective ones (around 11%). 5) Despite recent progress, accurately transcribing spontaneous speech in challenging acoustic environments remains difficult, even when using computationally intensive system ensembles.

Problem

• Improving multi-channel distant conversational speech recognition accuracy
• Enhancing neural speech separation for complex recording scenarios
• Optimizing diarization techniques for accurate speaker counting

Innovation

• End-to-end ASR systems with pre-trained models
• Guided source separation over neural SSE
• Diarization refinement via target-speaker techniques