Nonlinear dynamics of localization in neural receptive fields

2025-01-28

AI Summary

This work addresses why feedforward neural networks spontaneously develop localized receptive fields in the absence of explicit sparsity or independence constraints. Method: By rigorously deriving the effective learning dynamics of single neurons, we demonstrate that non-Gaussian higher-order statistics inherent in natural images directly drive the emergence of localization—without engineered coding objectives. Our approach integrates nonlinear dynamical systems analysis, effective-field theory, and statistical learning modeling, enabling quantitative prediction—based on natural image statistics—of the critical conditions and evolutionary trajectories for localization onset. Contribution/Results: Numerical simulations confirm robust replication of this mechanism in multi-neuron networks. The framework establishes a causal link between learning dynamics and receptive field structure, providing a novel theoretical foundation for understanding both biological visual cortex development and self-organization principles in artificial perceptual systems.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Learning with Manifolds
• Computer Vision: Representation Learning for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Social networks and social learning
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Localized receptive fields -- neurons that are selective for certain contiguous spatiotemporal features of their input -- populate early sensory regions of the mammalian brain. Unsupervised learning algorithms that optimize explicit sparsity or independence criteria replicate features of these localized receptive fields, but fail to explain directly how localization arises through learning without efficient coding, as occurs in early layers of deep neural networks and might occur in early sensory regions of biological systems. We consider an alternative model in which localized receptive fields emerge without explicit top-down efficiency constraints -- a feedforward neural network trained on a data model inspired by the structure of natural images. Previous work identified the importance of non-Gaussian statistics to localization in this setting but left open questions about the mechanisms driving dynamical emergence. We address these questions by deriving the effective learning dynamics for a single nonlinear neuron, making precise how higher-order statistical properties of the input data drive emergent localization, and we demonstrate that the predictions of these effective dynamics extend to the many-neuron setting. Our analysis provides an alternative explanation for the ubiquity of localization as resulting from the nonlinear dynamics of learning in neural circuits.

Problem

• Neural Networks
• Local Receptive Fields
• Non-Gaussian Statistics

Innovation

• Nonlinear Dynamics
• Local Receptive Fields
• Neural Network Learning