Cross-Demographic Portability of Deep NLP-Based Depression Models

2021-01-19

Spoken Language Technology Workshop

AI Summary

This study investigates the cross-age generalizability of deep NLP models for depression detection—specifically, their transferability from youth to older adult populations under severe age distribution skew in training data. Method: We employ end-to-end and feature-driven neural network architectures to extract depression-relevant linguistic patterns from speech or text and perform binary classification. Contribution/Results: To our knowledge, this is the first systematic evaluation of depression detection model robustness across real-world intergenerational cohorts. Our models achieve an AUC of 0.82 on youth test data and maintain strong performance (AUC = 0.76) when transferred to community-dwelling older adults. Crucially, stratifying older adults by longitudinal health stability reveals a subgroup with markedly improved performance (AUC = 0.81), demonstrating that individual-level stability is a key moderating factor. These findings provide empirical support and methodological guidance for age-inclusive depression screening.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Studies of user behavior, including longitudinal effects of personalized systems

Abstract

Deep learning models are rapidly gaining interest for real-world applications in behavioral health. An important gap in current literature is how well such models generalize over different populations. We study Natural Language Processing (NLP) based models to explore portability over two different corpora highly mismatched in age. The first and larger corpus contains younger speakers. It is used to train an NLP model to predict depression. When testing on unseen speakers from the same age distribution, this model performs at AUC=0.82. We then test this model on the second corpus, which comprises seniors from a retirement community. Despite the large demographic differences in the two corpora, we saw only modest degradation in performance for the senior-corpus data, achieving AUC=0.76. Interestingly, in the senior population, we find AUC=0.81 for the subset of patients whose health state is consistent over time. Implications for demographic portability of speech-based applications are discussed.

Problem

• Depression Detection
• Cross-age Group
• Language Technology

Innovation

• Natural Language Processing (NLP)
• Depression Detection
• Cross-age Robustness