Why all roads don't lead to Rome: Representation geometry varies across the human visual cortical hierarchy

2025-09-16

AI Summary

This study investigates the trade-off between coding efficiency and robustness in biological and artificial vision systems, specifically addressing whether scale-invariant representations constitute a universal property of the primate visual cortex. Method: Leveraging a population-geometry framework, we analyze representational structures in human fMRI data and artificial neural networks (ANNs), comparing self-supervised pretrained models with task-finetuned variants across systems. Contribution/Results: We find that primary visual cortex exhibits a power-law-decaying, scale-invariant singular spectrum—a universal geometric signature—whereas higher-order visual areas and finetuned ANNs significantly deviate from this structure. These deviations indicate that representational geometry is shaped by computational goals rather than by generic optimality principles. Our findings challenge the “universal optimal coding” hypothesis, demonstrating instead that representational properties emerge adaptively from functional demands. This work establishes a new paradigm for comparative analysis of neural coding principles across biological and artificial systems.

Technology Category

• Computer Vision: Representation Learning for Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Representation Learning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

Biological and artificial intelligence systems navigate the fundamental efficiency-robustness tradeoff for optimal encoding, i.e., they must efficiently encode numerous attributes of the input space while also being robust to noise. This challenge is particularly evident in hierarchical processing systems like the human brain. With a view towards understanding how systems navigate the efficiency-robustness tradeoff, we turned to a population geometry framework for analyzing representations in the human visual cortex alongside artificial neural networks (ANNs). In the ventral visual stream, we found general-purpose, scale-free representations characterized by a power law-decaying eigenspectrum in most areas. However, in certain higher-order visual areas did not have scale-free representations, indicating that scale-free geometry is not a universal property of the brain. In parallel, ANNs trained with a self-supervised learning objective also exhibited free-free geometry, but not after fine-tune on a specific task. Based on these empirical results and our analytical insights, we posit that a system's representation geometry is not a universal property and instead depends upon the computational objective.

Problem

• Analyzing representation geometry in visual cortex and ANNs
• Investigating efficiency-robustness tradeoff in hierarchical processing systems
• Examining how computational objectives shape representation geometry

Innovation

• Population geometry framework for analyzing representations
• Self-supervised learning in artificial neural networks
• Computational objective-dependent representation geometry