🤖 AI Summary
High inter-layer communication overhead severely limits the energy efficiency of artificial neural networks. Method: Inspired by nonlinear dendritic computation in biological neurons, we propose a differentiable dendritic computing unit that embeds local feature aggregation within individual neurons. Our approach integrates computational neuroscience modeling with end-to-end machine learning training to construct a dendrite-enhanced network architecture supporting gradient backpropagation. Contribution/Results: While dendritic nonlinearity alone does not enhance baseline learning capability, performing nonlinear aggregation of multi-branch inputs *within* each neuron significantly expands representational capacity and reduces inter-layer data transmission by up to 62% (on ImageNet using ResNet-18), without compromising inference accuracy. This work establishes a “communication-minimization” paradigm for neuromorphic accelerator design targeting low-power edge AI.
📝 Abstract
Our understanding of biological neuronal networks has profoundly influenced the development of artificial neural networks (ANNs). However, neurons utilized in ANNs differ considerably from their biological counterparts, primarily due to the absence of complex dendritic trees with local nonlinearities. Early studies have suggested that dendritic nonlinearities could substantially improve the learning capabilities of neural network models. In this study, we systematically examined the role of nonlinear dendrites within neural networks. Utilizing machine-learning methodologies, we assessed how dendritic nonlinearities influence neural network performance. Our findings demonstrate that dendritic nonlinearities do not substantially affect learning capacity; rather, their primary benefit lies in enabling network capacity expansion while minimizing communication costs through effective localized feature aggregation. This research provides critical insights with significant implications for designing future neural network accelerators aimed at reducing communication overhead during neural network training and inference.
