Practical Bayesian Inference for Speech SNNs: Uncertainty and Loss-Landscape Smoothing

2026-04-09

AI Summary

This work addresses the challenge of sharp and irregular prediction landscapes in spiking neural networks (SNNs) for speech processing, which arise from thresholding mechanisms and degrade model robustness and uncertainty estimation. For the first time, the study introduces an efficient Bayesian approach—improved variational online Newton (IVON)—into speech SNNs, applying probabilistic modeling to network weights to effectively smooth the loss landscape. By integrating surrogate gradient training with principled uncertainty quantification, the proposed method achieves substantial reductions in negative log-likelihood and Brier score on the Heidelberg Digits and Speech Commands datasets. Visualization via weight-space slicing further confirms smoother and more regular predictive behavior, demonstrating enhanced model calibration.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Speech
• Machine Learning: Bayesian Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

Spiking Neural Networks (SNNs) are naturally suited for speech processing tasks due to their specific dynamics, which allows them to handle temporal data. However, the threshold-based generation of spikes in SNNs intuitively causes an angular or irregular predictive landscape. We explore the effect of using the Bayesian learning approach for the weights on the irregular predictive landscape. For the surrogate-gradient SNNs, we also explore the application of the Improved Variational Online Newton (IVON) approach, which is an efficient variational approach. The performance of the proposed approach is evaluated on the Heidelberg Digits and Speech Commands datasets. The hypothesis is that the Bayesian approach will result in a smoother and more regular predictive landscape, given the angular nature of the deterministic predictive landscape. The experimental evaluation of the proposed approach shows improved performance on the negative log-likelihood and Brier score. Furthermore, the proposed approach has resulted in a smoother and more regular predictive landscape compared to the deterministic approach, based on the one-dimensional slices of the weight space

Problem

• Spiking Neural Networks
• predictive landscape
• Bayesian inference
• speech processing
• loss landscape

Innovation

• Bayesian inference
• Spiking Neural Networks
• loss-landscape smoothing
• IVON
• speech processing