From Activation to Causality: Discovery of Causal Visual Representations in the Human Brain

2026-05-22

AI Summary

Existing methods for localizing visual neural representations rely solely on activation strength, making them prone to confounding by visual or semantic distractors and resulting in numerous false positives. To address this limitation, this work proposes BrainCause, a novel framework that introduces causal inference into neural representation localization for the first time. By leveraging generative models to construct controlled stimulus sets—including target concepts, counterfactually edited variants, and distractors—and integrating image-to-fMRI encoding models with causal hypothesis testing, BrainCause identifies brain regions exhibiting concept-specific responses. The approach not only replicates established functional localizations but also uncovers dozens of new candidate representations. Its validity is confirmed through both predicted and empirical fMRI data, while further analysis reveals that, without causal validation, the majority of conventional localization results are likely false positives.

Abstract

Identifying which brain regions represent a visual concept in the human brain is a central challenge in neuroscience. Existing approaches have localized coarse functional regions (e.g., faces, places) through activation maximization, identifying regions that activate strongly for a target concept relative to other concepts. Yet strong activation alone does not establish that a region represents the concept itself, as responses may instead be driven by correlated visual or semantic cues. We introduce BrainCause, an automated framework that combines generative and brain models to synthesize controlled stimuli and validate neural representations through targeted causal testing. Given a query specifying a concept of interest, our framework constructs targeted stimulus sets comprising concept images, counterfactual edits that remove the target concept while preserving other image content, and images with candidate correlated distractors. It then uses an image-to-fMRI encoding model to predict brain responses and searches for representations that respond specifically to the target concept over correlated alternatives. BrainCause returns validated candidate representations and proposes follow-up fMRI experiments to further test or extend its discoveries. Our approach successfully recovers known functional localizations and identifies new candidate representations across dozens of concepts, validated on both predicted and measured fMRI data. Critically, we show that without causal validation, a large fraction of localizations would be false positives, confirming that activation alone is insufficient evidence of representation.

Problem

• causal representation
• visual concept
• brain localization
• activation vs. causality
• false positives

Innovation

• causal validation
• generative modeling
• fMRI encoding
• counterfactual stimuli
• visual representation