ToF-Splatting: Dense SLAM using Sparse Time-of-Flight Depth and Multi-Frame Integration

2025-04-23

AI Summary

Existing dense SLAM methods struggle to achieve robust performance on mobile and AR/VR devices under extreme power constraints, particularly when relying on ultra-sparse time-of-flight (ToF) depth measurements (e.g., tens of points per frame). To address this, this paper introduces 3D Gaussian Splatting into the SLAM framework for the first time, proposing an end-to-end dense reconstruction approach. Key contributions include: (1) a sparse ToF depth-guided 3D Gaussian representation; (2) a multi-frame fusion module that jointly leverages monocular images, sparse depth, and multi-view geometry under photometric-geometric consistency constraints; and (3) a geometric consistency optimization strategy to enhance pose and map stability. Evaluated on both synthetic and real-world ultra-sparse ToF datasets, our method achieves state-of-the-art performance in tracking accuracy, dense map completeness, and localization robustness—significantly outperforming prior approaches.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Time-of-Flight (ToF) sensors provide efficient active depth sensing at relatively low power budgets; among such designs, only very sparse measurements from low-resolution sensors are considered to meet the increasingly limited power constraints of mobile and AR/VR devices. However, such extreme sparsity levels limit the seamless usage of ToF depth in SLAM. In this work, we propose ToF-Splatting, the first 3D Gaussian Splatting-based SLAM pipeline tailored for using effectively very sparse ToF input data. Our approach improves upon the state of the art by introducing a multi-frame integration module, which produces dense depth maps by merging cues from extremely sparse ToF depth, monocular color, and multi-view geometry. Extensive experiments on both synthetic and real sparse ToF datasets demonstrate the viability of our approach, as it achieves state-of-the-art tracking and mapping performances on reference datasets.

Problem

• Enhancing SLAM with extremely sparse ToF depth data
• Integrating multi-frame sparse ToF, color, and geometry
• Achieving dense depth maps for mobile AR/VR devices

Innovation

• Uses sparse Time-of-Flight depth data
• Integrates multi-frame cues for dense maps
• Leverages 3D Gaussian Splatting for SLAM