ACEsplat: Accelerated 3D Gaussian Scene Regression via RGB and Poses Only

2026-06-20

AI Summary

This work addresses the limitations of existing 3D Gaussian splatting methods, which rely on external geometric priors—such as SfM point clouds or depth maps—and thus struggle to meet the demands for efficient and robust reconstruction in robotics and augmented reality applications. To overcome this, we propose ACEsplat, the first end-to-end framework that reconstructs 3D Gaussians using only RGB images and camera poses, without any external 3D priors. Our approach employs a two-stage pipeline: first, an internal geometric prior is established via self-supervised scene coordinate regression; then, a lightweight Gaussian initialization head combined with per-scene optimization enables efficient reconstruction. Trained in just 15–25 minutes on a single GPU, ACEsplat achieves PSNR scores of 29.11 dB and 33.20 dB on Wayspots and Cambridge Landmarks, respectively, and demonstrates superior novel view synthesis performance under the two-view setting of RealEstate10K.

Abstract

Per-scene 3D Gaussian Splatting (3DGS) enables high-fidelity rendering, but practical robotic and AR scene capture pipelines often depend on external geometric initialization (e.g., SfM point clouds or depth estimates), which can be slow and brittle in on-site deployment. We present ACEsplat, a fast per-scene optimization framework that reconstructs 3D Gaussian representations from RGB images and camera poses only, without requiring external 3D priors (e.g., precomputed SfM models or supervised depth maps). ACEsplat uses a two-stage pipeline: (1) a self-supervised scene coordinate regression (SCR) module builds an internal geometry prior within 4--5 minutes; (2) SCR features and coordinate priors are fused by a lightweight Gaussian initialization head, followed by per-scene 3DGS optimization. On static-view rendering, ACEsplat achieves 29.11 dB PSNR on Wayspots with real-time SLAM poses and 33.20 dB on Cambridge Landmarks with SfM-refined poses. On RealEstate10K sparse-view novel view synthesis, it achieves competitive image fidelity under a challenging 2-view setting. ACEsplat completes scene-specific SCR mapping and 3DGS reconstruction within 15--25 minutes on a single GPU, making it a practical RGB+pose-only solution for rapid scene setup in robotics and mixed-reality applications.

Problem

• 3D Gaussian Splatting
• scene reconstruction
• geometric initialization
• RGB and poses only
• external 3D priors

Innovation

• 3D Gaussian Splatting
• scene coordinate regression
• RGB-only reconstruction
• self-supervised geometry
• pose-guided rendering