Artificial Kuramoto Oscillatory Neurons

2024-10-17

arXiv.org

AI Summary

Neural networks suffer from limited dynamic spatiotemporal representation capabilities, hindering performance in unsupervised object discovery, adversarial robustness, uncertainty calibration, and logical reasoning. To address this, we propose Artificial Kuramoto Oscillatory Neurons (AKOrN)—the first work to systematically embed neural oscillation synchronization mechanisms into general-purpose neuron units. AKOrN replaces conventional threshold-based activation with phase dynamics and enables differentiable, dynamic binding among neurons via generalized Kuramoto coupling, maintaining full compatibility with mainstream architectures including fully connected networks, CNNs, and Transformers. The approach underscores the foundational role of dynamical synchronization in abstract representation learning, robust modeling, and uncertainty quantification. Extensive experiments demonstrate that AKOrN significantly outperforms baseline methods across unsupervised object discovery, adversarial robustness, calibrated uncertainty estimation, and logical reasoning tasks—validating both the efficacy and generalizability of oscillatory neural representations.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Neuro-Symbolic Learning
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Social networks and social learning
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

It has long been known in both neuroscience and AI that ``binding'' between neurons leads to a form of competitive learning where representations are compressed in order to represent more abstract concepts in deeper layers of the network. More recently, it was also hypothesized that dynamic (spatiotemporal) representations play an important role in both neuroscience and AI. Building on these ideas, we introduce Artificial Kuramoto Oscillatory Neurons (AKOrN) as a dynamical alternative to threshold units, which can be combined with arbitrary connectivity designs such as fully connected, convolutional, or attentive mechanisms. Our generalized Kuramoto updates bind neurons together through their synchronization dynamics. We show that this idea provides performance improvements across a wide spectrum of tasks such as unsupervised object discovery, adversarial robustness, calibrated uncertainty quantification, and reasoning. We believe that these empirical results show the importance of rethinking our assumptions at the most basic neuronal level of neural representation, and in particular show the importance of dynamical representations. Code: https://github.com/autonomousvision/akorn Project page: https://github.com/takerum/akorn_project_page

Problem

• Introduces dynamic Artificial Kuramoto Oscillatory Neurons
• Explores synchronization dynamics in neural representations
• Enhances performance in various AI and neuroscience tasks

Innovation

• Artificial Kuramoto Oscillatory Neurons
• Dynamic spatiotemporal representations
• Generalized Kuramoto synchronization dynamics