TrackerSplat: Exploiting Point Tracking for Fast and Robust Dynamic 3D Gaussians Reconstruction

2026-04-02

AI Summary

This work addresses temporal inconsistency and artifacts in existing 3D Gaussian splatting–based dynamic scene reconstruction methods under large inter-frame displacements. It introduces, for the first time, an off-the-shelf point tracking model into this framework to extract pixel trajectories and triangulate them into 3D Gaussian space, thereby guiding the initialization of Gaussian positions, rotations, and scales prior to training. This strategy effectively mitigates flickering and color shifts caused by large motions, significantly enhancing reconstruction robustness and rendering consistency. Combined with a multi-device parallelized multi-frame optimization scheme, the proposed system achieves superior reconstruction throughput on real-world datasets compared to current baselines while maintaining high-quality rendering fidelity.

Abstract

Recent advancements in 3D Gaussian Splatting (3DGS) have demonstrated its potential for efficient and photorealistic 3D reconstructions, which is crucial for diverse applications such as robotics and immersive media. However, current Gaussian-based methods for dynamic scene reconstruction struggle with large inter-frame displacements, leading to artifacts and temporal inconsistencies under fast object motions. To address this, we introduce \textit{TrackerSplat}, a novel method that integrates advanced point tracking methods to enhance the robustness and scalability of 3DGS for dynamic scene reconstruction. TrackerSplat utilizes off-the-shelf point tracking models to extract pixel trajectories and triangulate per-view pixel trajectories onto 3D Gaussians to guide the relocation, rotation, and scaling of Gaussians before training. This strategy effectively handles large displacements between frames, dramatically reducing the fading and recoloring artifacts prevalent in prior methods. By accurately positioning Gaussians prior to gradient-based optimization, TrackerSplat overcomes the quality degradation associated with large frame gaps when processing multiple adjacent frames in parallel across multiple devices, thereby boosting reconstruction throughput while preserving rendering quality. Experiments on real-world datasets confirm the robustness of TrackerSplat in challenging scenarios with significant displacements, achieving superior throughput under parallel settings and maintaining visual quality compared to baselines. The code is available at https://github.com/yindaheng98/TrackerSplat.

Problem

• Dynamic 3D Reconstruction
• 3D Gaussian Splatting
• Large Inter-frame Displacement
• Temporal Inconsistency
• Motion Artifacts

Innovation

• 3D Gaussian Splatting
• point tracking
• dynamic scene reconstruction
• temporal consistency
• parallel optimization