A compact neuromorphic system for ultra energy-efficient, on-device robot localization

2024-08-29

arXiv.org

AI Summary

To address the challenge of deploying high-accuracy, long-term visual place recognition (VPR) on resource-constrained edge robots—where conventional deep-learning-based VPR methods suffer from excessive computational overhead and energy consumption—this paper proposes the first fully neuromorphic, edge-deployable real-time place recognition system. Methodologically, we introduce LENS, a novel framework that tightly integrates spiking neural networks (SNNs), dynamic vision sensors (DVS), and the neuromorphic processor Speck, enabling event-driven, ultra-low-power feature encoding. The resulting model occupies only 180 KB with 44k parameters and achieves robust accuracy over an 8 km real-world trajectory. Power consumption is reduced by over 99% compared to state-of-the-art CNN-based approaches. The system has been successfully deployed on a hexapod robot, operating stably in real time. Our core contribution is establishing the first end-to-end neuromorphic localization paradigm tailored for energy- and compute-limited mobile robots.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Vision for Robotics & Autonomous Driving

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Neuromorphic computing offers a transformative pathway to overcome the computational and energy challenges faced in deploying robotic localization and navigation systems at the edge. Visual place recognition, a critical component for navigation, is often hampered by the high resource demands of conventional systems, making them unsuitable for small-scale robotic platforms which still require to perform complex, long-range tasks. Although neuromorphic approaches offer potential for greater efficiency, real-time edge deployment remains constrained by the complexity and limited scalability of bio-realistic networks. Here, we demonstrate a neuromorphic localization system that performs accurate place recognition in up to 8km of traversal using models as small as 180 KB with 44k parameters, while consuming less than 1% of the energy required by conventional methods. Our Locational Encoding with Neuromorphic Systems (LENS) integrates spiking neural networks, an event-based dynamic vision sensor, and a neuromorphic processor within a single SPECK(TM) chip, enabling real-time, energy-efficient localization on a hexapod robot. LENS represents the first fully neuromorphic localization system capable of large-scale, on-device deployment, setting a new benchmark for energy efficient robotic place recognition.

Problem

• Overcoming computational and energy challenges in robotic edge localization
• Enabling real-time, efficient localization for small-scale robotic platforms
• Reducing energy consumption while maintaining accurate place recognition

Innovation

• Neuromorphic computing for energy-efficient robot localization
• Spiking neural networks with event-based vision sensor
• Compact 180KB models for real-time edge deployment