Contravariance Theory: Strong Alignment for Minimal Solutions to Hard Tasks

2026-07-09

AI Summary

This work investigates whether deep neural networks and biological brains necessarily converge toward similar representational structures under high task difficulty. By constructing minimal-solution networks and introducing an affine-mapping-based representational alignment analysis—combined with hierarchical tracking and a formalized theory of contravariance—the study demonstrates that inter-network representational comparisons become insensitive to the choice of metric under strong task constraints. The core contribution lies in establishing theoretical guarantees bridging weak and strong alignment, revealing a bottom-up emergent “privileged axis” mechanism in end-to-end training. These findings provide a foundational theoretical framework for NeuroAI regarding the convergence of representations between artificial and biological systems.

Abstract

A series of results from the NeuroAI over the past fifteen years have raised core questions both about how to compare Deep Neural Network (DNN) models to the brain, and about how much convergent evolution to expect between artificial networks and real brain networks. Here, we show that for any two minimal DNN solutions to a sufficiently hard task: (i) "weak" alignment of network representations based on affine mappings guarantees "strong" alignment of privileged axes, and (ii) alignment "zippers" up the network hierarchy, causing the emergence of privileged axes from end-to-end task optimization. These results formalize the notion of contravariance from Cao and Yamins [2024], and illustrate important consequences for the theory of NeuroAI: with sufficiently strong tasks, choice of metric for inter-network comparison is not all that sensitive, and that convergent evolution is probably inevitable.

Problem

• alignment
• convergent evolution
• deep neural networks
• NeuroAI
• contravariance

Innovation

• contravariance
• strong alignment
• privileged axes
• convergent evolution
• NeuroAI