Including local feature interactions in deep non-negative matrix factorization networks improves performance

2025-03-26

AI Summary

This work addresses the lack of biological plausibility and limited capacity for local feature co-modeling in deep neural networks. We propose a novel deep Non-negative Matrix Factorization (NMF) network that, for the first time, embeds a local feature interaction module immediately after each NMF unit, enabling cortex-column-like excitatory–inhibitory cooperative computation under strict non-negativity constraints. The architecture integrates spiking stochastic computation with a lightweight convolutional backbone. Our key contributions are: (i) coupling local mixing mechanisms with the intrinsic positivity of NMF to jointly enhance interpretability and representational power; and (ii) eliminating biologically implausible negative synaptic weights inherent in conventional CNNs. Extensive experiments on multiple benchmark datasets demonstrate that our model significantly outperforms standard CNNs at comparable parameter counts, validating that local interactions under positivity constraints simultaneously improve both predictive performance and neurobiological consistency.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

The brain uses positive signals as a means of signaling. Forward interactions in the early visual cortex are also positive, realized by excitatory synapses. Only local interactions also include inhibition. Non-negative matrix factorization (NMF) captures the biological constraint of positive long-range interactions and can be implemented with stochastic spikes. While NMF can serve as an abstract formalization of early neural processing in the visual system, the performance of deep convolutional networks with NMF modules does not match that of CNNs of similar size. However, when the local NMF modules are each followed by a module that mixes the NMF's positive activities, the performances on the benchmark data exceed that of vanilla deep convolutional networks of similar size. This setting can be considered a biologically more plausible emulation of the processing in cortical (hyper-)columns with the potential to improve the performance of deep networks.

Problem

• Improving deep NMF networks with local feature interactions
• Enhancing performance via biologically plausible cortical emulation
• Combining NMF modules with mixing for better benchmark results

Innovation

• Incorporates local feature interactions in NMF
• Uses positive activities mixing post-NMF
• Emulates cortical column processing biologically