A Survey of fMRI to Image Reconstruction

2025-02-24

AI Summary

This paper addresses three core challenges in fMRI-to-image reconstruction (fMRI2Image): severe data scarcity, high inter-subject variability, and low semantic consistency. To tackle these, it proposes the first unified learning paradigm for fMRI2Image and introduces the field’s inaugural systematic taxonomy—categorizing methods into signal encoding, feature mapping, and image generation. Leveraging advances in deep learning, fMRI signal modeling, cross-modal alignment, and generative modeling, the work systematically analyzes the evolution of encoder–decoder architectures alongside diffusion- and GAN-based approaches. It identifies critical bottlenecks, establishes a standardized evaluation benchmark, and delineates three key research frontiers: interpretable modeling, neural–semantic representation alignment, and multi-subject generalization. The framework provides both theoretical foundations and practical guidance for brain–computer interfaces and computational neuroscience. (149 words)

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Multimodal Learning
• Computer Vision: Medical and Biological Imaging

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Functional magnetic resonance imaging (fMRI) based image reconstruction plays a pivotal role in decoding human perception, with applications in neuroscience and brain-computer interfaces. While recent advancements in deep learning and large-scale datasets have driven progress, challenges such as data scarcity, cross-subject variability, and low semantic consistency persist. To address these issues, we introduce the concept of fMRI-to-Image Learning (fMRI2Image) and present the first systematic review in this field. This review highlights key challenges, categorizes methodologies such as fMRI signal encoding, feature mapping, and image generator. Finally, promising research directions are proposed to advance this emerging frontier, providing a reference for future studies.

Problem

• fMRI to image reconstruction
• challenges in data and semantics
• systematic review of fMRI2Image

Innovation

• fMRI-to-Image Learning concept
• fMRI signal encoding methods
• feature mapping techniques