ACE-SLAM: Scene Coordinate Regression for Neural Implicit Real-Time SLAM

2025-12-15

AI Summary

This paper addresses the fundamental trade-off among real-time performance, memory efficiency, and relocalization accuracy in neural implicit SLAM systems. We propose the first real-time neural implicit RGB-D SLAM framework based on Scene Coordinate Regression (SCR). Our key contributions are: (i) the first integration of SCR into the neural SLAM pipeline as a lightweight, differentiable, and globally consistent implicit map representation; (ii) a compact, real-time-optimized SCR architecture enabling RGB-D multimodal feature fusion and robust mapping in dynamic environments; and (iii) end-to-end differentiable training with millisecond-scale pose optimization. Evaluated on both synthetic and real-world datasets, our method achieves state-of-the-art performance—exceeding 30 FPS, achieving relocalization latency under 5 ms, and reducing GPU memory consumption by over 60%—while ensuring low memory footprint, strong privacy preservation, and strict real-time constraints.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Learning to Search
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

We present a novel neural RGB-D Simultaneous Localization And Mapping (SLAM) system that learns an implicit map of the scene in real time. For the first time, we explore the use of Scene Coordinate Regression (SCR) as the core implicit map representation in a neural SLAM pipeline, a paradigm that trains a lightweight network to directly map 2D image features to 3D global coordinates. SCR networks provide efficient, low-memory 3D map representations, enable extremely fast relocalization, and inherently preserve privacy, making them particularly suitable for neural implicit SLAM. Our system is the first one to achieve strict real-time in neural implicit RGB-D SLAM by relying on a SCR-based representation. We introduce a novel SCR architecture specifically tailored for this purpose and detail the critical design choices required to integrate SCR into a live SLAM pipeline. The resulting framework is simple yet flexible, seamlessly supporting both sparse and dense features, and operates reliably in dynamic environments without special adaptation. We evaluate our approach on established synthetic and real-world benchmarks, demonstrating competitive performance against the state of the art. Project Page: https://github.com/ialzugaray/ace-slam

Problem

• Develops a real-time neural RGB-D SLAM system using implicit mapping.
• Introduces Scene Coordinate Regression for efficient, low-memory 3D map representation.
• Enables fast relocalization and operates reliably in dynamic environments.

Innovation

• Scene Coordinate Regression for implicit map representation
• Lightweight network mapping 2D features to 3D coordinates
• Real-time neural RGB-D SLAM with dynamic environment support