RABBiT: Rapidly adaptive BOLD foundation model via brain-tuning for accurate zero-shot and few-shot prediction of speech-elicited responses in the brain

2026-07-06

AI Summary

Existing models of language-evoked brain responses suffer from limited generalizability due to their dependence on individual subjects and specific stimuli. This work proposes a compact audio-to-fMRI encoder that integrates a pretrained speech backbone, region-specific attention mechanisms, and a shared–individual decomposition of brain responses, coupled with a parameter-efficient brain-tuning strategy. The approach achieves state-of-the-art zero-shot prediction performance across multiple unseen fMRI datasets involving 324 participants. Notably, with only ten minutes of individual subject data, it substantially outperforms conventional linear encoding models, demonstrating exceptional few-shot adaptation capability.

Abstract

Language understanding in the brain is context-dependent, varying across experimental stimuli and individuals, which makes it difficult to build computational models that generalize across both. This calls for a foundation model of language-evoked brain activity that can capture shared structure while adapting efficiently to new participants and inputs. We introduce RABBiT (Rapidly Adaptive BOLD foundation model via BraIn-Tuning), a compact audio-to-fMRI encoder designed for accurate zero- and few-shot prediction. A comprehensive evaluation on 324 participants across multiple unseen fMRI datasets shows that RABBiT enables accurate zero-shot prediction of fMRI responses to natural speech across auditory and language-selective regions, surpassing the SOTA foundation model for fMRI and predictions based on group averages. With as little as 10 minutes of participant-specific data, RABBiT further improves performance via parameter-efficient tuning, substantially outperforming per-participant linear models. RABBiT's performance is driven by two key innovations: (1) learned region-specific attention, and (2) a decomposition of brain responses into shared and subject-specific components, combined with a brain-tuned speech backbone. In addition to supporting strong predictive accuracy, the structured, region-specific representations that RABBiT learns enable interpretability. By eliminating the need for extensive per-participant data and model fitting, RABBiT enables scalable population-level analyses of language in the human brain. We make the code available at https://github.com/bridge-ai-neuro/rabbit.

Problem

• language-evoked brain activity
• zero-shot prediction
• few-shot prediction
• fMRI
• individual variability

Innovation

• region-specific attention
• shared and subject-specific decomposition
• brain-tuned speech backbone
• parameter-efficient tuning
• zero-shot fMRI prediction