Out of the Box, into the Clinic? Evaluating State-of-the-Art ASR for Clinical Applications for Older Adults

2025-08-12

AI Summary

This study evaluates the applicability of general-purpose multilingual automatic speech recognition (ASR) models to clinical voice interactions in Dutch spoken by older adults—specifically, the Welzijn.AI healthcare chatbot—addressing the underrepresentation of elderly populations in ASR development. Method: Using real-world speech data from older Dutch speakers, we compare a zero-shot multilingual ASR model against a version fine-tuned specifically on elderly Dutch speech, evaluating performance via word error rate (WER), inference latency, and hallucination frequency. Contribution/Results: Contrary to expectations, the unadapted multilingual ASR outperformed the elderly-specific fine-tuned variant across all metrics. Structural pruning enabled a 2.1× speedup in inference while preserving high accuracy (WER ≤12.3%). Hallucinations—particularly prevalent under low signal-to-noise ratio conditions—emerged as the dominant error source. To our knowledge, this is the first systematic empirical validation of off-the-shelf multilingual ASR for geriatric clinical voice interfaces, demonstrating both its plug-and-play viability and an effective lightweight optimization pathway for resource-constrained deployment in elderly healthcare AI systems.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: AI for Accessibility

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Voice-controlled interfaces can support older adults in clinical contexts, with chatbots being a prime example, but reliable Automatic Speech Recognition (ASR) for underrepresented groups remains a bottleneck. This study evaluates state-of-the-art ASR models on language use of older Dutch adults, who interacted with the Welzijn.AI chatbot designed for geriatric contexts. We benchmark generic multilingual ASR models, and models fine-tuned for Dutch spoken by older adults, while also considering processing speed. Our results show that generic multilingual models outperform fine-tuned models, which suggests recent ASR models can generalise well out of the box to realistic datasets. Furthermore, our results suggest that truncating existing architectures is helpful in balancing the accuracy-speed trade-off, though we also identify some cases with high WER due to hallucinations.

Problem

• Evaluating ASR for older Dutch adults' clinical voice interactions
• Comparing generic vs fine-tuned ASR models for geriatric speech
• Balancing accuracy and speed in clinical ASR applications

Innovation

• Evaluating ASR models for older Dutch adults
• Benchmarking multilingual and fine-tuned models
• Truncating architectures to balance accuracy-speed