Hierarchical Residuals Exploit Brain-Inspired Compositionality

2025-02-21

AI Summary

To address the limitations of deep convolutional neural networks—including restricted feature composition capability, slow convergence, and poor interpretability—this paper proposes brain-inspired Hierarchical Residual Networks (HiResNets). HiResNets are the first to incorporate cross-hierarchical direct connectivity mechanisms observed between cortical and subcortical regions in the mammalian brain into the residual learning framework. Specifically, they introduce multi-scale, long-range跨-layer residual connections within deep CNNs, integrated with hierarchical feature compression and reconstruction modules, thereby overcoming the representational bottlenecks inherent in conventional same-layer residual connections. Experimental results demonstrate that HiResNets significantly improve image classification accuracy and accelerate model convergence across multiple ResNet variants. Interpretability analysis further confirms their capacity for hierarchical semantic composition. Collectively, HiResNets establish a novel paradigm for designing biologically plausible deep learning architectures with enhanced functional and mechanistic fidelity to neurobiological principles.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Deep Neural Architectures and Foundation Models
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

We present Hierarchical Residual Networks (HiResNets), deep convolutional neural networks with long-range residual connections between layers at different hierarchical levels. HiResNets draw inspiration on the organization of the mammalian brain by replicating the direct connections from subcortical areas to the entire cortical hierarchy. We show that the inclusion of hierarchical residuals in several architectures, including ResNets, results in a boost in accuracy and faster learning. A detailed analysis of our models reveals that they perform hierarchical compositionality by learning feature maps relative to the compressed representations provided by the skip connections.

Problem

• Enhance deep convolutional neural networks
• Boost accuracy and learning speed
• Implement brain-inspired hierarchical compositionality

Innovation

• Hierarchical Residual Networks
• Long-range residual connections
• Brain-inspired compositionality