SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learning

2025-08-13

AI Summary

The mapping from visual stimuli to fMRI responses exhibits a one-to-many problem: identical images elicit heterogeneous hemodynamic responses across subjects, trials, or cognitive states, yet must preserve semantic-functional consistency. To address this, we propose SynBrain—a novel generative framework that jointly integrates BrainVAE with a semantics-guided neural mapper. SynBrain models the variability of neural responses via probabilistic latent-space modeling, explicitly enforcing semantic–neural functional consistency while preserving inter-subject and inter-trial biological diversity. Built upon variational autoencoding and semantics-constrained latent-space alignment, it supports few-shot adaptation and interpretable analysis. Experiments demonstrate that SynBrain significantly outperforms state-of-the-art methods on subject-specific visual–fMRI encoding and substantially improves fMRI decoding performance under data-limited conditions. Critically, the synthesized fMRI signals achieve both high fidelity and biological interpretability.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Humans and AI: Brain-Sensing and Analysis
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Deciphering how visual stimuli are transformed into cortical responses is a fundamental challenge in computational neuroscience. This visual-to-neural mapping is inherently a one-to-many relationship, as identical visual inputs reliably evoke variable hemodynamic responses across trials, contexts, and subjects. However, existing deterministic methods struggle to simultaneously model this biological variability while capturing the underlying functional consistency that encodes stimulus information. To address these limitations, we propose SynBrain, a generative framework that simulates the transformation from visual semantics to neural responses in a probabilistic and biologically interpretable manner. SynBrain introduces two key components: (i) BrainVAE models neural representations as continuous probability distributions via probabilistic learning while maintaining functional consistency through visual semantic constraints; (ii) A Semantic-to-Neural Mapper acts as a semantic transmission pathway, projecting visual semantics into the neural response manifold to facilitate high-fidelity fMRI synthesis. Experimental results demonstrate that SynBrain surpasses state-of-the-art methods in subject-specific visual-to-fMRI encoding performance. Furthermore, SynBrain adapts efficiently to new subjects with few-shot data and synthesizes high-quality fMRI signals that are effective in improving data-limited fMRI-to-image decoding performance. Beyond that, SynBrain reveals functional consistency across trials and subjects, with synthesized signals capturing interpretable patterns shaped by biological neural variability. The code will be made publicly available.

Problem

• Modeling variable hemodynamic responses to visual stimuli
• Capturing functional consistency in neural encoding
• Enhancing fMRI synthesis with probabilistic representation learning

Innovation

• Probabilistic learning models neural variability
• Semantic-to-Neural Mapper ensures high-fidelity synthesis
• Few-shot adaptation for new subjects efficiently