E-S2Feat:Semantic-Guided Spiking Local Feature Detection and Description for Event Cameras

2026-08-14

AI Summary

This study addresses the challenges of noise interference and the accuracy-efficiency trade-off in event camera feature extraction under resource constraints by proposing a semantics-guided spiking neural network framework. The method employs module-level spike activation to preserve fine-grained structures and integrates semantic prior modulation to enhance geometric stability and feature discriminability, thereby achieving joint optimization of feature representation and selection. Experimental results demonstrate that the proposed approach significantly outperforms baselines in pose estimation accuracy while matching conventional artificial neural networks. Furthermore, it achieves an approximate 4.8× improvement in theoretical energy efficiency. Validation within SLAM systems confirms its effectiveness for efficient and robust perception on edge devices, offering a promising solution for neuromorphic vision applications in resource-limited environments.

Abstract

Benefiting from high temporal resolution and dynamic range, event-based local feature methods have attracted increasing attention. However, event sparsity, noise, and limited texture still hinder robust local feature learning. Deploying such methods on resource-constrained platforms such as unmanned aerial vehicles also requires balancing accuracy and energy efficiency. To address these challenges, this paper proposes \textbf{E-S2Feat}, a spiking neural network framework for event-based local feature detection and description. The framework jointly optimizes local feature learning from the perspectives of feature representation and selection. First, a module-specific spiking activation mechanism preserves fine-grained structural cues and discriminative information under low-bit, energy-efficient inference, thereby improving overall representation fidelity. Furthermore, a semantic-guided feature modulation mechanism leverages semantic priors to refine keypoint response distributions and enhance local descriptor discriminability, thereby guiding the model to extract local features with greater geometric stability and stronger discriminative capability. Experiments on the ECD and EDS datasets show that the proposed method significantly outperforms baseline methods such as SuperEvent in pose estimation accuracy. It also achieves accuracy comparable to its artificial neural network counterpart while delivering an approximately 4.8-fold improvement in theoretical computational energy efficiency. Visual-inertial odometry experiments on the TUM-VIE dataset further verify the effectiveness and practical application potential of the proposed method in complete SLAM systems.

Problem

• Event Cameras
• Local Feature Detection
• Spiking Neural Networks
• Energy Efficiency
• Robustness

Innovation

• Spiking Neural Network
• Semantic-Guided Feature Modulation
• Event Camera
• Local Feature Detection and Description
• Energy Efficiency