MVP: Multi-source Voice Pathology detection

2025-05-26

AI Summary

Automatic diagnosis of voice disorders is hindered by the scarcity of pathological speech data and the heterogeneity across multi-source recordings (e.g., sentence reading, sustained vowel phonation). To address this, we propose the first raw-waveform-based multi-source speech fusion framework. Our method introduces a novel intermediate-layer feature fusion strategy to enable cross-lingual complementary modeling of pathological characteristics; it employs an end-to-end Transformer architecture that directly processes waveforms, integrating three hierarchical fusion mechanisms: waveform concatenation, intermediate feature fusion, and decision-level fusion. Evaluated on German, Portuguese, and Italian datasets, our approach achieves up to a 13% improvement in AUC over single-source baselines. This work establishes a scalable, non-invasive solution for low-resource, multi-source voice pathology detection.

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Speech
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Voice disorders significantly impact patient quality of life, yet non-invasive automated diagnosis remains under-explored due to both the scarcity of pathological voice data, and the variability in recording sources. This work introduces MVP (Multi-source Voice Pathology detection), a novel approach that leverages transformers operating directly on raw voice signals. We explore three fusion strategies to combine sentence reading and sustained vowel recordings: waveform concatenation, intermediate feature fusion, and decision-level combination. Empirical validation across the German, Portuguese, and Italian languages shows that intermediate feature fusion using transformers best captures the complementary characteristics of both recording types. Our approach achieves up to +13% AUC improvement over single-source methods.

Problem

• Detects voice pathology from multi-source recordings
• Addresses scarcity and variability in pathological voice data
• Improves diagnosis accuracy using transformer-based fusion strategies

Innovation

• Transformers process raw voice signals directly
• Three fusion strategies combine different recordings
• Intermediate feature fusion optimizes performance best