Statistical Learning for Latent Embedding Alignment with Application to Brain Encoding and Decoding

2026-03-21

AI Summary

This work addresses the challenge of low sample efficiency in fMRI-based encoding and decoding models, which stems from the scarcity of paired fMRI–stimulus data and substantial inter-subject variability. To overcome this, the authors propose a lightweight latent embedding alignment framework that operates with frozen pre-trained encoders and decoders. By leveraging abundant unpaired stimulus embeddings through a reverse semi-supervised learning strategy, the method introduces a novel meta-transfer mechanism that integrates residual debiasing and sparse aggregation to enable effective cross-subject knowledge transfer and alignment refinement. Theoretical analysis provides generalization bounds and safety guarantees under limited sample regimes. Extensive experiments on large-scale fMRI image reconstruction benchmarks demonstrate significant improvements in both sample efficiency and reconstruction performance, confirming the approach’s effectiveness and robustness.

Technology Category

• Machine Learning: Multimodal Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Brain encoding and decoding aims to understand the relationship between external stimuli and brain activities, and is a fundamental problem in neuroscience. In this article, we study latent embedding alignment for brain encoding and decoding, with a focus on improving sample efficiency under limited fMRI-stimulus paired data and substantial subject heterogeneity. We propose a lightweight alignment framework equipped with two statistical learning components: inverse semi-supervised learning that leverages abundant unpaired stimulus embeddings through inverse mapping and residual debiasing, and meta transfer learning that borrows strength from pretrained models across subjects via sparse aggregation and residual correction. Both methods operate exclusively at the alignment stage while keeping encoders and decoders frozen, allowing for efficient computation, modular deployment, and rigorous theoretical analysis. We establish finite-sample generalization bounds and safety guarantees, and demonstrate competitive empirical performance on the large-scale fMRI-image reconstruction benchmark data.

Problem

• brain encoding
• brain decoding
• latent embedding alignment
• sample efficiency
• subject heterogeneity

Innovation

• latent embedding alignment
• inverse semi-supervised learning
• meta transfer learning
• sample efficiency
• fMRI decoding