6DOPE-GS: Online 6D Object Pose Estimation using Gaussian Splatting

2024-12-02

arXiv.org

AI Summary

Accurate, real-time 6D pose estimation and tracking of objects in streaming RGB-D video remains challenging without explicit object models. Method: We propose an online optimization framework integrating Gaussian rasterization, featuring (i) the first incremental 2D Gaussian splatting with dynamic keyframe selection; (ii) an adaptive Gaussian density pruning strategy based on opacity statistics, balancing training stability and real-time performance; and (iii) joint optimization of object pose and implicit 3D reconstruction. Contributions/Results: Our method achieves state-of-the-art accuracy on HO3D and YCBInEOAT benchmarks while accelerating inference by 5×. To our knowledge, it is the first approach enabling real-time, robust tracking and dense reconstruction of dynamic objects using a single RGB-D camera—significantly broadening spatial coverage and enhancing practical deployability in real-world scenarios.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Efficient and accurate object pose estimation is an essential component for modern vision systems in many applications such as Augmented Reality, autonomous driving, and robotics. While research in model-based 6D object pose estimation has delivered promising results, model-free methods are hindered by the high computational load in rendering and inferring consistent poses of arbitrary objects in a live RGB-D video stream. To address this issue, we present 6DOPE-GS, a novel method for online 6D object pose estimation &tracking with a single RGB-D camera by effectively leveraging advances in Gaussian Splatting. Thanks to the fast differentiable rendering capabilities of Gaussian Splatting, 6DOPE-GS can simultaneously optimize for 6D object poses and 3D object reconstruction. To achieve the necessary efficiency and accuracy for live tracking, our method uses incremental 2D Gaussian Splatting with an intelligent dynamic keyframe selection procedure to achieve high spatial object coverage and prevent erroneous pose updates. We also propose an opacity statistic-based pruning mechanism for adaptive Gaussian density control, to ensure training stability and efficiency. We evaluate our method on the HO3D and YCBInEOAT datasets and show that 6DOPE-GS matches the performance of state-of-the-art baselines for model-free simultaneous 6D pose tracking and reconstruction while providing a 5$ imes$ speedup. We also demonstrate the method's suitability for live, dynamic object tracking and reconstruction in a real-world setting.

Problem

• Online 6D object pose estimation in RGB-D video
• Model-free pose tracking with Gaussian Splatting
• Efficient dynamic object reconstruction and tracking

Innovation

• Uses Gaussian Splatting for fast differentiable rendering
• Implements dynamic keyframe selection for accuracy
• Applies opacity-based pruning for adaptive density control