Efficiency-Performance Trade-offs in Neural Speaker Diarization via Structured Pruning and Low-Bit Quantization

2026-06-11

AI Summary

Deploying streaming speaker diarization systems on resource-constrained devices requires careful trade-offs among model size, latency, and performance, particularly in time-sensitive healthcare dispatch scenarios. This work presents the first systematic evaluation of buffering strategies, latency constraints, and model compression techniques for speaker diarization on real-world streaming speech from the SIMSAMU dataset. The authors apply structured pruning and FP16/low-bit quantization to compress neural segmentation models and analyze their real-time factor and diarization error rate (DER) within a streaming inference framework. Experimental results show that FP16 quantization halves model size with negligible impact on real-time performance but increases DER by 40% relative to the baseline. The study further reveals that additional buffering does not necessarily benefit low-latency operation and delineates practical deployment boundaries for time-critical applications.

Abstract

Streaming speaker diarization is crucial for time-critical medical dispatch, but deploying it on resource-constrained hardware requires smaller, faster models. Using SIMSAMU, a dataset of simulated medical-dispatch conversations, we evaluate streaming behavior before compressing the segmentation model with pruning and low-bit quantization. We characterize performance across a range of streaming latency budgets and find that additional buffering is not consistently beneficial, while very low-latency operating points can substantially degrade performance. Our study shows that model compression trades performance for memory footprint, and we highlight an operating point where FP16 reduces model size by half with essentially unchanged real-time factor, at a cost of a 40\% relative DER increase against the baseline. This work characterizes the trade-offs for real-time deployment and contributes to speech technology that can enable reliable human communication in time-critical contexts.

Problem

• speaker diarization
• model compression
• efficiency-performance trade-off
• streaming latency
• resource-constrained deployment

Innovation

• structured pruning
• low-bit quantization
• streaming speaker diarization
• latency-performance trade-off
• model compression