Reading Between the Lines: Combining Pause Dynamics and Semantic Coherence for Automated Assessment of Thought Disorder

2025-07-17

AI Summary

Traditional clinical rating scales for assessing formal thought disorder (FTD) in schizophrenia-spectrum disorders are resource-intensive and difficult to scale. Existing automated speech analysis approaches predominantly rely on single-modality features and fail to jointly model temporal dynamics (e.g., pause patterns) and semantic coherence. To address this, we propose the first systematic multimodal framework integrating ASR-derived pause dynamics—such as pause frequency and duration distribution—with semantic coherence metrics computed via pretrained language models. The framework is designed to be robust across diverse clinical contexts. We employ support vector regression (SVR) for feature fusion and evaluation on the TOPSY dataset yields a correlation coefficient of ρ = 0.649 for FTD severity prediction and an AUC of 83.71% for detecting severe cases—both significantly outperforming unimodal baselines. This work establishes a scalable, objective, and quantitatively grounded methodology for automated assessment of speech disorganization in psychosis.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Multimodal Learning
• Cognitive Modeling & Cognitive Systems: Applications

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Formal thought disorder (FTD), a hallmark of schizophrenia spectrum disorders, manifests as incoherent speech and poses challenges for clinical assessment. Traditional clinical rating scales, though validated, are resource-intensive and lack scalability. Automated speech analysis with automatic speech recognition (ASR) allows for objective quantification of linguistic and temporal features of speech, offering scalable alternatives. The use of utterance timestamps in ASR captures pause dynamics, which are thought to reflect the cognitive processes underlying speech production. However, the utility of integrating these ASR-derived features for assessing FTD severity requires further evaluation. This study integrates pause features with semantic coherence metrics across three datasets: naturalistic self-recorded diaries (AVH, n = 140), structured picture descriptions (TOPSY, n = 72), and dream narratives (PsyCL, n = 43). We evaluated pause related features alongside established coherence measures, using support vector regression (SVR) to predict clinical FTD scores. Key findings demonstrate that pause features alone robustly predict the severity of FTD. Integrating pause features with semantic coherence metrics enhanced predictive performance compared to semantic-only models, with integration of independent models achieving correlations up to {ho} = 0.649 and AUC = 83.71% for severe cases detection (TOPSY, with best {ho} = 0.584 and AUC = 79.23% for semantic-only models). The performance gains from semantic and pause features integration held consistently across all contexts, though the nature of pause patterns was dataset-dependent. These findings suggest that frameworks combining temporal and semantic analyses provide a roadmap for refining the assessment of disorganized speech and advance automated speech analysis in psychosis.

Problem

• Assessing thought disorder severity using automated speech analysis
• Integrating pause dynamics with semantic coherence for FTD evaluation
• Enhancing predictive performance of FTD detection across diverse datasets

Innovation

• Combining pause dynamics with semantic coherence metrics
• Using support vector regression for FTD prediction
• Integrating ASR-derived pause and semantic features