Self-organized MT Direction Maps Emerge from Spatiotemporal Contrastive Optimization

2026-05-12

AI Summary

This study investigates the computational origins of direction-selective maps in the primate visual dorsal stream, specifically in the middle temporal (MT) area. By integrating spatiotemporal contrastive learning with biologically inspired spatial regularization, we train a 3D ResNet on natural videos and, for the first time, achieve self-organized mechanisms of both ventral and dorsal pathways within a unified framework. The model spontaneously develops brain-like MT direction maps and topographic organization through a trade-off between discriminative task pressure and spatial smoothness. Crucially, it successfully reproduces key physiological properties observed in macaque MT—including direction selectivity index, circular variance, and pinwheel density—with remarkable fidelity to in vivo experimental data, thereby revealing the computational foundations underlying direction selectivity.

Abstract

The spatial and functional organization of the primate visual cortex is a fundamental problem in neuroscience. While recent computational frameworks like the Topographic Deep Artificial Neural Network (TDANN) have successfully modeled spatial organization in the ventral stream, the computational origins of the dorsal stream's distinct topographies, such as direction-selective maps in the middle temporal (MT) area, remain largely unresolved. In this work, we present a spatiotemporal TDANN to investigate whether MT topography is governed by the same universal principles. By training a 3D ResNet on naturalistic videos via a Momentum Contrast (MoCo) self-supervised paradigm alongside a biologically inspired spatial loss, we demonstrate the spontaneous emergence of brain-like direction maps and topological pinwheel structures. Crucially, we reveal that MT tuning properties, characterized by strong direction selectivity paired with a residual axial component, arise from a strict optimization trade-off between task-driven discriminative pressure and spatial regularization. The model's representations quantitatively match in vivo macaque MT physiological baselines, including direction selectivity index, circular variance, and pinwheel density. These findings unify the computational origins of the ventral and dorsal streams, establishing a general mechanism for cortical self-organization.

Problem

• MT direction maps
• dorsal stream
• cortical self-organization
• spatiotemporal contrastive optimization
• topographic organization

Innovation

• spatiotemporal contrastive learning
• topographic self-organization
• direction-selective maps
• MT area modeling
• spatial regularization