Speaker Retrieval in the Wild: Challenges, Effectiveness and Robustness

2025-04-26

AI Summary

This work addresses the speaker retrieval challenge in large-scale legacy audiovisual archives (e.g., BBC Rewind), where sparse metadata and highly variable acoustic conditions—such as noise, reverberation, and bandwidth limitations—severely hinder performance. We propose the first end-to-end, “in-the-wild” speaker retrieval framework specifically designed for such uncontrolled archival settings. Our method jointly optimizes speaker diarization, embedding extraction, and query construction under zero- or weakly-supervised conditions. Key innovations include distortion-robust training, metadata-guided pseudo-label generation, and a multi-granularity evaluation protocol. By integrating self-supervised speech embeddings (ECAPA-TDNN) with speaker diarization outputs, the framework significantly improves retrieval accuracy and generalization in fully unsupervised scenarios. Extensive evaluation on real degraded archival recordings demonstrates strong robustness across diverse acoustic distortions, stable retrieval performance, and seamless transferability to other large-scale, unconstrained audiovisual corpora.

Technology Category

• Natural Language Processing: Speech
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Computer Vision: Image and Video Retrieval

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

There is a growing abundance of publicly available or company-owned audio/video archives, highlighting the increasing importance of efficient access to desired content and information retrieval from these archives. This paper investigates the challenges, solutions, effectiveness, and robustness of speaker retrieval systems developed"in the wild"which involves addressing two primary challenges: extraction of task-relevant labels from limited metadata for system development and evaluation, as well as the unconstrained acoustic conditions encountered in the archive, ranging from quiet studios to adverse noisy environments. While we focus on the publicly-available BBC Rewind archive (spanning 1948 to 1979), our framework addresses the broader issue of speaker retrieval on extensive and possibly aged archives with no control over the content and acoustic conditions. Typically, these archives offer a brief and general file description, mostly inadequate for specific applications like speaker retrieval, and manual annotation of such large-scale archives is unfeasible. We explore various aspects of system development (e.g., speaker diarisation, embedding extraction, query selection) and analyse the challenges, possible solutions, and their functionality. To evaluate the performance, we conduct systematic experiments in both clean setup and against various distortions simulating real-world applications. Our findings demonstrate the effectiveness and robustness of the developed speaker retrieval systems, establishing the versatility and scalability of the proposed framework for a wide range of applications beyond the BBC Rewind corpus.

Problem

• Addressing speaker retrieval challenges in unconstrained audio archives
• Developing robust systems with limited metadata and varied acoustic conditions
• Evaluating performance in clean and distorted real-world scenarios

Innovation

• Extracts task-relevant labels from limited metadata
• Handles unconstrained acoustic conditions effectively
• Uses scalable framework for large aged archives