Sparse components distinguish visual pathways & their alignment to neural networks

2025-10-09

AI Summary

While the human visual cortex exhibits pronounced functional specialization across its ventral, dorsal, and lateral pathways, single-task-trained deep neural networks (DNNs) surprisingly achieve broad representational alignment with neural responses across all three pathways—giving rise to a “functional specificity vs. model generality” representational alignment paradox. Method: We propose Sparse Component Alignment (SCA), a method that leverages fMRI data and DNN representations to disentangle pathway-specific dominant functional components via sparse decomposition, enabling fine-grained, cross-system representational alignment. Contribution/Results: Unlike conventional group-level geometric analyses, SCA uncovers latent neural tuning structures, substantially enhancing the spatial resolution and sensitivity of brain–model representational matching. We find that standard vision DNNs align best with the ventral stream, whereas the dorsal and lateral streams harbor distinct representational dimensions inadequately captured by current models. This work establishes a novel paradigm and quantitative framework for investigating visual pathway functional segregation and for developing biologically plausible artificial vision systems.

Technology Category

• Computer Vision: Representation Learning for Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The ventral, dorsal, and lateral streams in high-level human visual cortex are implicated in distinct functional processes. Yet, deep neural networks (DNNs) trained on a single task model the entire visual system surprisingly well, hinting at common computational principles across these pathways. To explore this inconsistency, we applied a novel sparse decomposition approach to identify the dominant components of visual representations within each stream. Consistent with traditional neuroscience research, we find a clear difference in component response profiles across the three visual streams -- identifying components selective for faces, places, bodies, text, and food in the ventral stream; social interactions, implied motion, and hand actions in the lateral stream; and some less interpretable components in the dorsal stream. Building on this, we introduce Sparse Component Alignment (SCA), a new method for measuring representational alignment between brains and machines that better captures the latent neural tuning of these two visual systems. Using SCA, we find that standard visual DNNs are more aligned with the ventral than either dorsal or lateral representations. SCA reveals these distinctions with greater resolution than conventional population-level geometry, offering a measure of representational alignment that is sensitive to a system's underlying axes of neural tuning.

Problem

• Identifying distinct visual pathway components using sparse decomposition
• Measuring brain-machine alignment with Sparse Component Alignment method
• Comparing neural tuning between ventral, dorsal, and lateral visual streams

Innovation

• Sparse decomposition identifies dominant visual components
• SCA method measures brain-machine representational alignment
• SCA reveals ventral stream alignment with DNNs