Efficient Spiking Point Mamba for Point Cloud Analysis

2025-04-19

AI Summary

To address the weak long-range dependency modeling capability and high energy consumption of existing 3D Spiking Neural Networks (SNNs), this paper pioneers the integration of the state-space model Mamba into the spiking computation paradigm, proposing a sparsified 3D Mamba architecture. Key contributions include: (1) Hierarchical Dynamic Encoding (HDE), enabling efficient spike-based spatiotemporal representation of point cloud sequences; (2) the Spiking Mamba Block (SMB), synergistically combining SNNs’ temporal coding capacity with Mamba’s powerful sequential modeling; and (3) an asymmetric SNN-ANN pretraining-finetuning paradigm to mitigate training difficulties inherent to spiking networks. Evaluated on three variants of ScanObjectNN, our method achieves absolute accuracy gains of 6.2–7.4%; on ShapeNetPart, it improves instance-level mIOU by 1.9%. Moreover, it reduces energy consumption by over 3.5× compared to its ANN counterpart.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Bio-inspired Spiking Neural Networks (SNNs) provide an energy-efficient way to extract 3D spatio-temporal features. However, existing 3D SNNs have struggled with long-range dependencies until the recent emergence of Mamba, which offers superior computational efficiency and sequence modeling capability. In this work, we propose Spiking Point Mamba (SPM), the first Mamba-based SNN in the 3D domain. Due to the poor performance of simply transferring Mamba to 3D SNNs, SPM is designed to utilize both the sequence modeling capabilities of Mamba and the temporal feature extraction of SNNs. Specifically, we first introduce Hierarchical Dynamic Encoding (HDE), an improved direct encoding method that effectively introduces dynamic temporal mechanism, thereby facilitating temporal interactions. Then, we propose a Spiking Mamba Block (SMB), which builds upon Mamba while learning inter-time-step features and minimizing information loss caused by spikes. Finally, to further enhance model performance, we adopt an asymmetric SNN-ANN architecture for spike-based pre-training and finetune. Compared with the previous state-of-the-art SNN models, SPM improves OA by +6.2%, +6.1%, and +7.4% on three variants of ScanObjectNN, and boosts instance mIOU by +1.9% on ShapeNetPart. Meanwhile, its energy consumption is at least 3.5x lower than that of its ANN counterpart. The code will be made publicly available.

Problem

• Enhance 3D point cloud analysis using energy-efficient Spiking Neural Networks
• Address long-range dependency challenges in 3D Spiking Neural Networks
• Combine Mamba's sequence modeling with SNNs' temporal feature extraction

Innovation

• Hierarchical Dynamic Encoding for temporal interactions
• Spiking Mamba Block for inter-time-step features
• Asymmetric SNN-ANN architecture for pre-training