THE-SEAN: A Heart Rate Variation-Inspired Temporally High-Order Event-Based Visual Odometry with Self-Supervised Spiking Event Accumulation Networks

2025-03-07

AI Summary

Existing event-camera-based visual odometry methods typically employ fixed-frequency triggering, leading to computational redundancy and noise accumulation. To address this, we propose a motion-adaptive dynamic estimation triggering mechanism that models trigger decisions as a learnable process. Specifically, we introduce, for the first time, a biologically inspired self-supervised spiking neural network (SNN)—motivated by cardiac rhythm regulation—integrated with block-matching point rewards, trace feedback from the Fisher information matrix, and spatiotemporal feature extraction from event streams to enable high-order temporal triggering modeling. Our approach eliminates predefined timing schedules, enabling trigger frequency to dynamically adapt to both ego-motion states and scene complexity. Evaluated on public benchmarks, our method achieves a 13% improvement in trajectory accuracy, a 9% enhancement in trajectory smoothness, and a 38% gain in triggering efficiency over state-of-the-art approaches.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Event-based visual odometry has recently gained attention for its high accuracy and real-time performance in fast-motion systems. Unlike traditional synchronous estimators that rely on constant-frequency (zero-order) triggers, event-based visual odometry can actively accumulate information to generate temporally high-order estimation triggers. However, existing methods primarily focus on adaptive event representation after estimation triggers, neglecting the decision-making process for efficient temporal triggering itself. This oversight leads to the computational redundancy and noise accumulation. In this paper, we introduce a temporally high-order event-based visual odometry with spiking event accumulation networks (THE-SEAN). To the best of our knowledge, it is the first event-based visual odometry capable of dynamically adjusting its estimation trigger decision in response to motion and environmental changes. Inspired by biological systems that regulate hormone secretion to modulate heart rate, a self-supervised spiking neural network is designed to generate estimation triggers. This spiking network extracts temporal features to produce triggers, with rewards based on block matching points and Fisher information matrix (FIM) trace acquired from the estimator itself. Finally, THE-SEAN is evaluated across several open datasets, thereby demonstrating average improvements of 13% in estimation accuracy, 9% in smoothness, and 38% in triggering efficiency compared to the state-of-the-art methods.

Problem

• Dynamic adjustment of estimation triggers in event-based visual odometry.
• Reduction of computational redundancy and noise accumulation.
• Improvement in accuracy, smoothness, and triggering efficiency.

Innovation

• Dynamic adjustment of estimation triggers
• Self-supervised spiking neural network
• Improved accuracy and triggering efficiency