Spiking Neural Networks for Continuous Control via End-to-End Model-Based Learning

2025-09-03

AI Summary

To address the limited applicability of Spiking Neural Networks (SNNs) in continuous motor control, this paper proposes an end-to-end, fully spiking model-based reinforcement learning framework. The method integrates Leaky Integrate-and-Fire (LIF) neuron dynamics with surrogate gradient learning to jointly optimize a learnable forward dynamics model and a spiking policy network. To enhance training stability, it introduces learnable initial time constants and dedicated regularization strategies. Evaluated on 2D planar grasping and 6-DOF robotic arm torque control—both high-dimensional, nonlinear continuous-control tasks—the framework achieves stable convergence and high-precision continuous action generation. This work constitutes the first demonstration of effective modeling and control of complex continuous dynamical systems using a fully spiking architecture, thereby establishing a novel paradigm for brain-inspired intelligent robotic control.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Intelligent Robots: Behavior Learning & Control
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Despite recent progress in training spiking neural networks (SNNs) for classification, their application to continuous motor control remains limited. Here, we demonstrate that fully spiking architectures can be trained end-to-end to control robotic arms with multiple degrees of freedom in continuous environments. Our predictive-control framework combines Leaky Integrate-and-Fire dynamics with surrogate gradients, jointly optimizing a forward model for dynamics prediction and a policy network for goal-directed action. We evaluate this approach on both a planar 2D reaching task and a simulated 6-DOF Franka Emika Panda robot. Results show that SNNs can achieve stable training and accurate torque control, establishing their viability for high-dimensional motor tasks. An extensive ablation study highlights the role of initialization, learnable time constants, and regularization in shaping training dynamics. We conclude that while stable and effective control can be achieved, recurrent spiking networks remain highly sensitive to hyperparameter settings, underscoring the importance of principled design choices.

Problem

• Training spiking neural networks for continuous motor control tasks
• Developing end-to-end model-based learning for robotic arm control
• Achieving stable training and accurate torque control with SNNs

Innovation

• End-to-end model-based learning for control
• Combines Leaky Integrate-and-Fire with surrogate gradients
• Jointly optimizes forward model and policy network