Real Time Scheduling Framework for Multi Object Detection via Spiking Neural Networks

2025-01-29

AI Summary

Spiking Neural Networks (SNNs) deployed on energy-constrained autonomous mobile platforms—such as autonomous vehicles—face fundamental trade-offs between temporal determinism, recognition reliability, and energy efficiency. Existing SNN inference schedulers lack rigorous real-time guarantees and suffer from significant accuracy degradation under resource constraints. Method: This paper proposes RT-SNN, the first real-time–aware, high-accuracy multi-object detection scheduling framework for SNNs. It introduces a novel membrane-potential reuse mechanism and an adjustable time-step execution strategy to jointly optimize timing predictability and spike-based inference fidelity. We further propose a joint modeling approach combining membrane confidence and mean absolute error, and design a two-phase scheduling framework comprising offline schedulability analysis and online membrane-confidence–driven adaptation. Results: Evaluated on Spiking-YOLO, RT-SNN strictly satisfies hard real-time constraints (R1), incurs <1.2% mAP loss, and reduces energy consumption by >47%, significantly outperforming state-of-the-art SNN inference schedulers.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Hardware-aware ML

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Given the energy constraints in autonomous mobile agents (AMAs), such as unmanned vehicles, spiking neural networks (SNNs) are increasingly favored as a more efficient alternative to traditional artificial neural networks. AMAs employ multi-object detection (MOD) from multiple cameras to identify nearby objects while ensuring two essential objectives, (R1) timing guarantee and (R2) high accuracy for safety. In this paper, we propose RT-SNN, the first system design, aiming at achieving R1 and R2 in SNN-based MOD systems on AMAs. Leveraging the characteristic that SNNs gather feature data of input image termed as membrane potential, through iterative computation over multiple timesteps, RT-SNN provides multiple execution options with adjustable timesteps and a novel method for reusing membrane potential to support R1. Then, it captures how these execution strategies influence R2 by introducing a novel notion of mean absolute error and membrane confidence. Further, RT-SNN develops a new scheduling framework consisting of offline schedulability analysis for R1 and a run-time scheduling algorithm for R2 using the notion of membrane confidence. We deployed RT-SNN to Spiking-YOLO, the SNN-based MOD model derived from ANN-to-SNN conversion, and our experimental evaluation confirms its effectiveness in meeting the R1 and R2 requirements while providing significant energy efficiency.

Problem

• Spiking Neural Networks
• Multi-Object Detection
• Autonomous Vehicles

Innovation

• RT-SNN
• Spike-based Multi-object Detection
• Energy Efficiency