Heterogeneous Neural Predictivity from Language Models During Naturalistic Comprehension

2026-06-25

AI Summary

This study investigates whether representations from language models can effectively predict neural activity during natural language comprehension and disentangles their predictive power from homologies in underlying neural computations. By leveraging frozen pretrained language models to generate high-dimensional semantic features and integrating multimodal neuroimaging (MEG/ECoG), source-level analysis, block-wise encoding models, and rigorously matched control experiments, the authors systematically evaluate the models’ ability to predict neural responses during naturalistic listening and reading tasks. Results demonstrate that a significant proportion of channels in both the Brain Treebank and Podcast ECoG datasets meet stringent prediction criteria. Language model features substantially outperform low-level baselines, with individual-level predictive advantages exhibiting localized topography—providing the first multimodal evidence that language model representations effectively annotate the neural dynamics of natural language understanding.

Abstract

Language-model representations provide structured, high-dimensional annotations of naturalistic language stimuli and can serve as informative neural predictors during comprehension. We analyzed locked derived data from Brain Treebank, MEG-MASC, and Podcast ECoG with eight frozen language models, blocked encoding models, and matched temporal, nuisance, and representation-capacity controls. Positive held-out prediction and gains over low-level baselines were widespread in source-level summaries. Across Brain Treebank and Podcast ECoG, 67 of 432 evaluable rows met a controlled predictive-only criterion, and model-side feature ablations changed prediction scores in most evaluable source rows. Brain-derived, timing-linked, acoustic, and implanted-signal controls confirmed component-level sensitivity of the analysis pipeline. These findings show that language-model-derived quantities can annotate neural activity during natural speech and text comprehension. Participant-level matched-control advantages were localized rather than uniform, response-profile and feature-specificity contrasts bounded representational or computational interpretations, and complete co-indexed integrated interpretation will require future jointly indexed coverage. Together, the analyses identify language-model features as useful neural predictors and separate predictive usefulness from claims about shared neural organization or language-processing computations.

Problem

• language models
• neural predictivity
• naturalistic comprehension
• neural encoding
• representational similarity

Innovation

• language models
• neural prediction
• naturalistic comprehension
• encoding models
• feature ablation