NeuroXVocal: Detection and Explanation of Alzheimer's Disease through Non-invasive Analysis of Picture-prompted Speech

2025-02-14

AI Summary

Non-invasive early diagnosis of Alzheimer’s disease (AD) remains challenging. This work proposes the first classification–explanation co-designed framework for AD speech screening: a multimodal Transformer fuses acoustic features, ASR-derived transcripts, and pretrained language embeddings for high-accuracy detection; and retrieval-augmented generation (RAG) is innovatively integrated with a curated AD medical literature knowledge base to generate clinically credible, interpretable explanations of speech biomarkers. Evaluated on the ADReSSo 2021 dataset, the framework achieves 95.77% accuracy—setting a new state-of-the-art. Clinical expert assessment confirms that the generated explanations exhibit significant medical relevance and practical utility. To our knowledge, this is the first study to deeply embed RAG into the AD speech diagnostic pipeline, simultaneously optimizing discriminative performance and model interpretability. The approach advances the development of non-invasive, trustworthy AD screening tools suitable for real-world clinical deployment.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Machine Learning: Multimodal Learning
• Natural Language Processing: Speech

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

The early diagnosis of Alzheimer's Disease (AD) through non invasive methods remains a significant healthcare challenge. We present NeuroXVocal, a novel dual-component system that not only classifies but also explains potential AD cases through speech analysis. The classification component (Neuro) processes three distinct data streams: acoustic features capturing speech patterns and voice characteristics, textual features extracted from speech transcriptions, and precomputed embeddings representing linguistic patterns. These streams are fused through a custom transformer-based architecture that enables robust cross-modal interactions. The explainability component (XVocal) implements a Retrieval-Augmented Generation (RAG) approach, leveraging Large Language Models combined with a domain-specific knowledge base of AD research literature. This architecture enables XVocal to retrieve relevant clinical studies and research findings to generate evidence-based context-sensitive explanations of the acoustic and linguistic markers identified in patient speech. Using the IS2021 ADReSSo Challenge benchmark dataset, our system achieved state-of-the-art performance with 95.77% accuracy in AD classification, significantly outperforming previous approaches. The explainability component was qualitatively evaluated using a structured questionnaire completed by medical professionals, validating its clinical relevance. NeuroXVocal's unique combination of high-accuracy classification and interpretable, literature-grounded explanations demonstrates its potential as a practical tool for supporting clinical AD diagnosis.

Problem

• Early Alzheimer's detection via speech analysis
• Classify and explain AD through multimodal data
• Enhance AD diagnosis with evidence-based explanations

Innovation

• Transformer-based cross-modal fusion
• Retrieval-Augmented Generation (RAG) approach
• Non-invasive speech analysis for AD