Application-oriented automatic hyperparameter optimization for spiking neural network prototyping

2025-02-13

AI Summary

Manual hyperparameter tuning for Spiking Neural Networks (SNNs) suffers from low efficiency and unstable performance due to complex neuronal dynamics and strong inter-parameter coupling. Method: This paper proposes the first application-oriented SNN hyperparameter optimization (HPO) paradigm. We design an SNN-specific optimization pipeline built upon the NNI framework, integrating Bayesian optimization with early stopping, and ensuring compatibility with mainstream simulators—including SpikingJelly and BindsNET—for task-driven, end-to-end hyperparameter search. Contribution/Results: We systematically introduce application-centric HPO to the SNN domain for the first time; enable cross-task transferable optimization; and achieve average accuracy gains of 8.2%–14.7% across multiple benchmark tasks, while reducing tuning time by over 60%.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Search and Optimization: Learning to Search
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Hyperparameter optimization (HPO) is of paramount importance in the development of high-performance, specialized artificial intelligence (AI) models, ranging from well-established machine learning (ML) solutions to the deep learning (DL) domain and the field of spiking neural networks (SNNs). The latter introduce further complexity due to the neuronal computational units and their additional hyperparameters, whose inadequate setting can dramatically impact the final model performance. At the cost of possible reduced generalization capabilities, the most suitable strategy to fully disclose the power of SNNs is to adopt an application-oriented approach and perform extensive HPO experiments. To facilitate these operations, automatic pipelines are fundamental, and their configuration is crucial. In this document, the Neural Network Intelligence (NNI) toolkit is used as reference framework to present one such solution, with a use case example providing evidence of the corresponding results. In addition, a summary of published works employing the presented pipeline is reported as possible source of insights into application-oriented HPO experiments for SNN prototyping.

Problem

• Automates hyperparameter optimization for spiking neural networks
• Addresses complexity in SNN hyperparameter settings
• Uses NNI toolkit for application-oriented HPO experiments

Innovation

• Automatic hyperparameter optimization for SNNs
• Application-oriented approach in HPO
• NNI toolkit for SNN prototyping