CognoSpeak: an automatic, remote assessment of early cognitive decline in real-world conversational speech

2025-01-10

AI Summary

Conventional clinical assessments for early cognitive decline are costly, inconvenient, and lack scalability; existing speech-based approaches suffer from limited data volume and shallow feature representation. Method: We developed the first remote, automated early cognitive decline assessment system specifically designed for real-world conversational speech. The system deploys a mobile/web-based virtual agent to administer standardized clinical tasks—including memory probing, verbal fluency, and picture description—while synchronously capturing multimodal audio-video and clinical metadata. Contribution/Results: By integrating clinician-informed task design with large-scale, ecologically valid speech acquisition, the system enables unobtrusive home-based screening. Leveraging a lightweight DistilBERT architecture, multi-task speech modeling, and cross-source metadata fusion, it achieves an F1-score of 0.873 on a cohort of 126 participants, accurately differentiating dementia, mild cognitive impairment, and cognitively healthy individuals—demonstrating strong ecological validity and efficacy of non-invasive remote screening.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Natural Language Processing: Speech
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The early signs of cognitive decline are often noticeable in conversational speech, and identifying those signs is crucial in dealing with later and more serious stages of neurodegenerative diseases. Clinical detection is costly and time-consuming and although there has been recent progress in the automatic detection of speech-based cues, those systems are trained on relatively small databases, lacking detailed metadata and demographic information. This paper presents CognoSpeak and its associated data collection efforts. CognoSpeak asks memory-probing long and short-term questions and administers standard cognitive tasks such as verbal and semantic fluency and picture description using a virtual agent on a mobile or web platform. In addition, it collects multimodal data such as audio and video along with a rich set of metadata from primary and secondary care, memory clinics and remote settings like people's homes. Here, we present results from 126 subjects whose audio was manually transcribed. Several classic classifiers, as well as large language model-based classifiers, have been investigated and evaluated across the different types of prompts. We demonstrate a high level of performance; in particular, we achieved an F1-score of 0.873 using a DistilBERT model to discriminate people with cognitive impairment (dementia and people with mild cognitive impairment (MCI)) from healthy volunteers using the memory responses, fluency tasks and cookie theft picture description. CognoSpeak is an automatic, remote, low-cost, repeatable, non-invasive and less stressful alternative to existing clinical cognitive assessments.

Problem

• Early Dementia Detection
• Remote Monitoring
• Automated Diagnosis

Innovation

• CognoSpeak
• DistilBERT
• Remote Cognitive Assessment