Automated Extraction of Spatio-Semantic Graphs for Identifying Cognitive Impairment

2025-02-02

AI Summary

Current cognitive-linguistic impairment assessment methods rely on eye-tracking data or manual annotation of Content Units (CIUs) to construct spatio-semantic graphs, limiting clinical accessibility and scalability. This paper proposes the first end-to-end automated method for constructing spatio-semantic graphs: leveraging NLP-based semantic parsing and spatial relation modeling, integrated with cognitive-linguistic constraint rules, it directly extracts CIUs and generates graph structures from descriptive narratives of the Cookie Theft picture—without requiring eye-tracking signals or manual annotations. The approach significantly improves deployment efficiency and clinical feasibility. Evaluated on distinguishing individuals with cognitive impairment from healthy controls, its automatically derived graph features achieve superior classification performance, with statistically significant gains over manually annotated baselines and enhanced discriminative power across clinical populations.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Natural Language Processing: Information Extraction
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Existing methods for analyzing linguistic content from picture descriptions for assessment of cognitive-linguistic impairment often overlook the participant's visual narrative path, which typically requires eye tracking to assess. Spatio-semantic graphs are a useful tool for analyzing this narrative path from transcripts alone, however they are limited by the need for manual tagging of content information units (CIUs). In this paper, we propose an automated approach for estimation of spatio-semantic graphs (via automated extraction of CIUs) from the Cookie Theft picture commonly used in cognitive-linguistic analyses. The method enables the automatic characterization of the visual semantic path during picture description. Experiments demonstrate that the automatic spatio-semantic graphs effectively differentiate between cognitively impaired and unimpaired speakers. Statistical analyses reveal that the features derived by the automated method produce comparable results to the manual method, with even greater group differences between clinical groups of interest. These results highlight the potential of the automated approach for extracting spatio-semantic features in developing clinical speech models for cognitive impairment assessment.

Problem

• Automated extraction of spatio-semantic graphs
• Cognitive impairment assessment from picture descriptions
• Comparison with manual tagging methods

Innovation

• Automated extraction of spatio-semantic graphs
• Automatic characterization of visual semantic path
• Statistical comparison with manual method results