Region Masking to Accelerate Video Processing on Neuromorphic Hardware

2025-03-21

AI Summary

To address the high energy consumption and latency caused by spike redundancy in Spiking Neural Network (SNN)-based video processing on neuromorphic edge devices, this paper proposes a dynamic region masking mechanism at the input stage. A lightweight, spatial-attention-inspired mask module is integrated into the SNN’s first layer to identify and suppress semantically insignificant regions in real time, thereby inhibiting spurious spike generation at the source. This work is the first to embed region masking directly into the SNN input layer, achieving a favorable trade-off between hardware efficiency and accuracy controllability. Implemented on the Intel Loihi 2 platform with sigma-delta event encoding and hardware-aware scheduling, the method achieves a 1.65× reduction in energy-delay product under 60% region masking, with only a 1.09% drop in mAP@0.5. Throughput and real-time performance are significantly enhanced.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Video Understanding & Activity Analysis

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The rapidly growing demand for on-chip edge intelligence on resource-constrained devices has motivated approaches to reduce energy and latency of deep learning models. Spiking neural networks (SNNs) have gained particular interest due to their promise to reduce energy consumption using event-based processing. We assert that while sigma-delta encoding in SNNs can take advantage of the temporal redundancy across video frames, they still involve a significant amount of redundant computations due to processing insignificant events. In this paper, we propose a region masking strategy that identifies regions of interest at the input of the SNN, thereby eliminating computation and data movement for events arising from unimportant regions. Our approach demonstrates that masking regions at the input not only significantly reduces the overall spiking activity of the network, but also provides significant improvement in throughput and latency. We apply region masking during video object detection on Loihi 2, demonstrating that masking approximately 60% of input regions can reduce energy-delay product by 1.65x over a baseline sigma-delta network, with a degradation in mAP@0.5 by 1.09%.

Problem

• Reducing redundant computations in spiking neural networks for video processing
• Improving energy and latency efficiency in neuromorphic hardware
• Enhancing video object detection by masking insignificant input regions

Innovation

• Region masking reduces redundant SNN computations
• Focuses on input regions of interest
• Improves throughput, latency, and energy efficiency