A Multidisciplinary AI Board for Multimodal Dementia Characterization and Risk Assessment

2026-03-23

AI Summary

This work proposes Cerebra, an interpretable, multimodal multi-agent AI system designed to integrate seamlessly into clinical workflows. Addressing the challenges posed by heterogeneous, dynamic, and often incomplete patient data in real-world practice, Cerebra synergistically fuses electronic health records, clinical notes, and medical imaging to enable robust inference even under missing modalities while supporting privacy-preserving deployment. The system empowers clinicians through structured representation learning, visual analytics, and conversational interaction for real-time risk assessment and decision-making. Evaluated on a cohort of 3 million patients, Cerebra achieves strong performance in dementia risk prediction (AUROC=0.80), diagnosis (AUROC=0.86), and survival prediction (C-index=0.81). Notably, clinician diagnostic accuracy improves by 17.5 percentage points when aided by the system.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Modern clinical practice increasingly depends on reasoning over heterogeneous, evolving, and incomplete patient data. Although recent advances in multimodal foundation models have improved performance on various clinical tasks, most existing models remain static, opaque, and poorly aligned with real-world clinical workflows. We present Cerebra, an interactive multi-agent AI team that coordinates specialized agents for EHR, clinical notes, and medical imaging analysis. These outputs are synthesized into a clinician-facing dashboard that combines visual analytics with a conversational interface, enabling clinicians to interrogate predictions and contextualize risk at the point of care. Cerebra supports privacy-preserving deployment by operating on structured representations and remains robust when modalities are incomplete. We evaluated Cerebra using a massive multi-institutional dataset spanning 3 million patients from four independent healthcare systems. Cerebra consistently outperformed both state-of-the-art single-modality models and large multimodal language model baselines. In dementia risk prediction, it achieved AUROCs up to 0.80, compared with 0.74 for the strongest single-modality model and 0.68 for language model baselines. For dementia diagnosis, it achieved an AUROC of 0.86, and for survival prediction, a C-index of 0.81. In a reader study with experienced physicians, Cerebra significantly improved expert performance, increasing accuracy by 17.5 percentage points in prospective dementia risk estimation. These results demonstrate Cerebra's potential for interpretable, robust decision support in clinical care.

Problem

• multimodal dementia characterization
• risk assessment
• heterogeneous clinical data
• incomplete data
• clinical decision support

Innovation

• multi-agent AI
• multimodal fusion
• interpretable clinical decision support
• privacy-preserving deployment
• robustness to missing modalities