MK-SGC-SC: Multiple Kernel Guided Sparse Graph Construction in Spectral Clustering for Unsupervised Speaker Diarization

2026-01-24

AI Summary

This work addresses the challenges of lacking labeled data and reliance on pretraining in unsupervised speaker diarization by proposing a multi-kernel fusion–based similarity measure. It systematically integrates polynomial kernels with first-order arc-cosine kernels for the first time to construct a sparse affinity graph that emphasizes local structural properties, followed by spectral clustering for speaker segmentation and clustering. The method requires no supervision or pretrained models and achieves state-of-the-art unsupervised performance on major benchmarks including DIHARD-III, AMI, and VoxConverse, significantly advancing the practical applicability of unsupervised speaker diarization.

Technology Category

• Machine Learning: Kernel Methods
• Natural Language Processing: Speech
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Speaker diarization aims to segment audio recordings into regions corresponding to individual speakers. Although unsupervised speaker diarization is inherently challenging, the prospect of identifying speaker regions without pretraining or weak supervision motivates research on clustering techniques. In this work, we share the notable observation that measuring multiple kernel similarities of speaker embeddings to thereafter craft a sparse graph for spectral clustering in a principled manner is sufficient to achieve state-of-the-art performances in a fully unsupervised setting. Specifically, we consider four polynomial kernels and a degree one arccosine kernel to measure similarities in speaker embeddings, using which sparse graphs are constructed in a principled manner to emphasize local similarities. Experiments show the proposed approach excels in unsupervised speaker diarization over a variety of challenging environments in the DIHARD-III, AMI, and VoxConverse corpora. To encourage further research, our implementations are available at https://github.com/nikhilraghav29/MK-SGC-SC.

Problem

• unsupervised speaker diarization
• spectral clustering
• sparse graph construction
• speaker embeddings
• multiple kernel similarity

Innovation

• Multiple Kernel Learning
• Sparse Graph Construction
• Spectral Clustering
• Unsupervised Speaker Diarization
• Speaker Embeddings