Towards explainable reference-free speech intelligibility evaluation of people with pathological speech

2026-02-13

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Evaluation and Analysis

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Objective assessment of speech that reflects meaningful changes in communication is crucial for clinical decision making and reproducible research. While existing objective assessments, particularly reference-based approaches, can capture intelligibility changes, they are often hindered by lack of explainability and the need for labor-intensive manual transcriptions. To address these issues, this work proposes the reference-free, explainable ASR Inconsistency Score. We evaluate this method on pathological speech in Dutch, Spanish and English, and compare its performance to a reference-based Word Error Rate (WER) baseline. Our results demonstrate that the ASR Inconsistency Score achieves a high correlation with expert perceptual ratings, with performance closely matching, and in one case exceeding, a standard reference-based Word Error Rate (WER) baseline.

Problem

• speech intelligibility
• pathological speech
• reference-free evaluation
• explainability
• objective assessment

Innovation

• reference-free
• explainable AI
• speech intelligibility
• pathological speech
• ASR Inconsistency Score