CogPic: A Multimodal Dataset for Early Cognitive Impairment Assessment via Picture Description Tasks

2026-04-02

AI Summary

Current research on automated cognitive impairment assessment is hindered by the scarcity of large-scale, rigorously synchronized, and clinically validated multimodal datasets. To address this gap, this work introduces CogPic—the largest and most comprehensive multimodal benchmark dataset to date—collected during a naturalistic picture description task from 574 participants, with simultaneously recorded audio, visual, and linguistic data. Clinical labels were established through multiple rounds of consensus diagnosis by neuropsychological experts, ensuring high diagnostic rigor. CogPic not only fills a critical void in high-quality multimodal resources but also demonstrates, through benchmark experiments, its foundational utility in developing robust, unbiased, and clinically generalizable models for automatic cognitive assessment.

Abstract

The automated evaluation of cognitive status utilizing multimedia technologies presents a promising frontier in early dementia diagnosis. However, the development of robust machine learning models for cognitive impairment detection is frequently hindered by the scarcity of large-scale, strictly synchronized, and clinically validated multimodal datasets. To bridge this critical gap, we introduce the CogPic database, a comprehensive multimodal benchmark meticulously designed for fine-grained cognitive impairment detection. The dataset comprises strictly synchronized audio, visual, and linguistic data continuously collected from 574 participants during a naturalistic picture description task. To establish highly reliable diagnostic ground truth, expert clinical neuropsychologists conducted exhaustive evaluations, stratifying participants into distinct cognitive groups through a comprehensive clinical consensus. Consequently, CogPic stands as the largest, most modality-rich, and most meticulously evaluated dataset of its kind to date. By conducting extensive benchmark experiments on the CogPic dataset, we establish an exceptionally robust, unbiased, and clinically generalizable foundation to propel future multimedia research in automated cognitive health assessment. Detailed information and access application procedures for our CogPic database are available at https://cogpic.github.io/.

Problem

• cognitive impairment
• multimodal dataset
• early dementia diagnosis
• picture description task
• clinical validation

Innovation

• multimodal dataset
• cognitive impairment assessment
• picture description task
• clinical validation
• synchronized audio-visual-linguistic data