Autonomous Reinforcement Learning Robot Control with Intel's Loihi 2 Neuromorphic Hardware

2025-12-03

AI Summary

This work addresses the challenge of deploying reinforcement learning (RL)-trained artificial neural networks (ANNs) efficiently on neuromorphic hardware for real-time control of space and terrestrial robots. We propose a lossless conversion method that directly maps purely simulation-trained ReLU-ANN policies to spiking Sigma-Delta neural networks (SDNNs), enabling deployment on the Intel Loihi 2 neuromorphic chip. We validate the approach via closed-loop motion control of the Astrobee free-flying robot in NVIDIA Omniverse Isaac Lab, achieving millisecond-level latency and ultra-low-power inference. Experiments demonstrate that Loihi 2 matches GPU-based control performance while delivering substantial energy efficiency gains. To our knowledge, this is the first demonstration that a purely simulation-trained RL policy can be deployed on neuromorphic hardware—without fine-tuning—to enable real-time autonomous control. The results establish a novel, high-efficiency, low-latency computing paradigm for resource-constrained robotic applications in space and remote environments.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We present an end-to-end pipeline for deploying reinforcement learning (RL) trained Artificial Neural Networks (ANNs) on neuromorphic hardware by converting them into spiking Sigma-Delta Neural Networks (SDNNs). We demonstrate that an ANN policy trained entirely in simulation can be transformed into an SDNN compatible with Intel's Loihi 2 architecture, enabling low-latency and energy-efficient inference. As a test case, we use an RL policy for controlling the Astrobee free-flying robot, similar to a previously hardware in space-validated controller. The policy, trained with Rectified Linear Units (ReLUs), is converted to an SDNN and deployed on Intel's Loihi 2, then evaluated in NVIDIA's Omniverse Isaac Lab simulation environment for closed-loop control of Astrobee's motion. We compare execution performance between GPU and Loihi 2. The results highlight the feasibility of using neuromorphic platforms for robotic control and establish a pathway toward energy-efficient, real-time neuromorphic computation in future space and terrestrial robotics applications.

Problem

• Deploy RL-trained ANNs on neuromorphic hardware via SDNN conversion
• Enable low-latency, energy-efficient inference for robotic control applications
• Establish pathway for neuromorphic computation in space and terrestrial robotics

Innovation

• Convert ANN to SDNN for neuromorphic hardware
• Deploy RL policy on Intel Loihi 2 chip
• Enable low-latency energy-efficient robotic control