Multi-Sensory Cognitive Computing for Learning Population-level Brain Connectivity

2025-08-15

AI Summary

Existing connectivity-based brain template (CBT) methods suffer from poor interpretability, high computational cost, and neglect of cognitive function modeling. To address these limitations, we propose mCOCO—a novel framework that integrates multisensory inputs into reservoir computing (RC) for the first time, enabling construction of cognitively grounded population-level functional brain connectivity templates. mCOCO jointly models structural, functional, and cognitive dimensions by synergistically leveraging dynamic fMRI BOLD signals, multimodal neuroimaging data, and cognition-informed dynamical systems modeling. Compared to state-of-the-art graph neural network (GNN)-based approaches, mCOCO-generated templates demonstrate significant improvements in node centrality estimation, subject discriminability, topological plausibility, and multisensory memory retention—while maintaining high computational efficiency and strong interpretability. The implementation is publicly available.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Multimodal Learning
• Humans and AI: Brain-Sensing and Analysis

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

The generation of connectional brain templates (CBTs) has recently garnered significant attention for its potential to identify unique connectivity patterns shared across individuals. However, existing methods for CBT learning such as conventional machine learning and graph neural networks (GNNs) are hindered by several limitations. These include: (i) poor interpretability due to their black-box nature, (ii) high computational cost, and (iii) an exclusive focus on structure and topology, overlooking the cognitive capacity of the generated CBT. To address these challenges, we introduce mCOCO (multi-sensory COgnitive COmputing), a novel framework that leverages Reservoir Computing (RC) to learn population-level functional CBT from BOLD (Blood-Oxygen-level-Dependent) signals. RC's dynamic system properties allow for tracking state changes over time, enhancing interpretability and enabling the modeling of brain-like dynamics, as demonstrated in prior literature. By integrating multi-sensory inputs (e.g., text, audio, and visual data), mCOCO captures not only structure and topology but also how brain regions process information and adapt to cognitive tasks such as sensory processing, all in a computationally efficient manner. Our mCOCO framework consists of two phases: (1) mapping BOLD signals into the reservoir to derive individual functional connectomes, which are then aggregated into a group-level CBT - an approach, to the best of our knowledge, not previously explored in functional connectivity studies - and (2) incorporating multi-sensory inputs through a cognitive reservoir, endowing the CBT with cognitive traits. Extensive evaluations show that our mCOCO-based template significantly outperforms GNN-based CBT in terms of centeredness, discriminativeness, topological soundness, and multi-sensory memory retention. Our source code is available at https://github.com/basiralab/mCOCO.

Problem

• Improves interpretability of brain connectivity templates
• Reduces computational cost in CBT learning
• Integrates cognitive traits with structural connectivity

Innovation

• Reservoir Computing for brain connectivity modeling
• Multi-sensory inputs enhance cognitive trait capture
• Dynamic system properties improve interpretability efficiency