RTGS: Real-Time 3D Gaussian Splatting SLAM via Multi-Level Redundancy Reduction

2025-10-08

AI Summary

To address the challenge of real-time execution of 3D Gaussian Splatting (3DGS)-based SLAM on edge devices, this work proposes a holistic hardware–software co-design for lightweight acceleration. Algorithmically, it introduces adaptive Gaussian pruning, dynamic pixel downsampling, and keyframe reuse to reduce computational redundancy. Hardware-wise, it designs a sub-tile streaming scheduler, R&B (reduction-and-broadcast) cache, and gradient merging unit, while optimizing memory access patterns and enabling backward-pass gradient reuse. Collectively, these techniques eliminate redundant computation and memory traffic. The resulting system achieves ≥30 FPS real-time performance on edge GPUs, with an 82.5× improvement in energy efficiency, while preserving rendering quality without statistically significant degradation. To the best of our knowledge, this is the first system-level acceleration framework for efficient 3DGS-based SLAM tailored specifically for edge deployment.

Technology Category

• Machine Learning: Hardware-aware ML
• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

3D Gaussian Splatting (3DGS) based Simultaneous Localization and Mapping (SLAM) systems can largely benefit from 3DGS's state-of-the-art rendering efficiency and accuracy, but have not yet been adopted in resource-constrained edge devices due to insufficient speed. Addressing this, we identify notable redundancies across the SLAM pipeline for acceleration. While conceptually straightforward, practical approaches are required to minimize the overhead associated with identifying and eliminating these redundancies. In response, we propose RTGS, an algorithm-hardware co-design framework that comprehensively reduces the redundancies for real-time 3DGS-SLAM on edge. To minimize the overhead, RTGS fully leverages the characteristics of the 3DGS-SLAM pipeline. On the algorithm side, we introduce (1) an adaptive Gaussian pruning step to remove the redundant Gaussians by reusing gradients computed during backpropagation; and (2) a dynamic downsampling technique that directly reuses the keyframe identification and alpha computing steps to eliminate redundant pixels. On the hardware side, we propose (1) a subtile-level streaming strategy and a pixel-level pairwise scheduling strategy that mitigates workload imbalance via a Workload Scheduling Unit (WSU) guided by previous iteration information; (2) a Rendering and Backpropagation (R&B) Buffer that accelerates the rendering backpropagation by reusing intermediate data computed during rendering; and (3) a Gradient Merging Unit (GMU) to reduce intensive memory accesses caused by atomic operations while enabling pipelined aggregation. Integrated into an edge GPU, RTGS achieves real-time performance (>= 30 FPS) on four datasets and three algorithms, with up to 82.5x energy efficiency over the baseline and negligible quality loss. Code is available at https://github.com/UMN-ZhaoLab/RTGS.

Problem

• Accelerating 3D Gaussian Splatting SLAM for real-time edge device performance
• Reducing computational redundancies across the SLAM pipeline for efficiency
• Minimizing overhead in identifying and eliminating pipeline redundancies

Innovation

• Adaptive Gaussian pruning reuses backpropagation gradients
• Dynamic downsampling eliminates redundant pixels via keyframes
• Hardware co-design enables real-time edge SLAM acceleration