You Need a Transition Plane: Bridging Continuous Panoramic 3D Reconstruction with Perspective Gaussian Splatting

2025-04-12

AI Summary

In 3D Gaussian Splatting (3DGS) reconstruction from a single panoramic image, severe distortion in equirectangular projection and boundary discontinuities in cubic mapping impede robust Gaussian optimization. To address this, we propose a spherical-consistent 3DGS reconstruction framework. Our method introduces: (1) a transition-plane mechanism to mitigate abrupt Gaussian orientation changes at cube-face boundaries; (2) a synergistic optimization strategy combining intra-face local refinement with global spherical-space fine-tuning; and (3) spherical-coordinate sampling and perspective-aware Gaussian rendering to eliminate stitching artifacts and enhance geometric continuity. Evaluated on diverse benchmarks—including indoor/outdoor, selfie-style, and mobile-captured panoramas—our approach consistently outperforms state-of-the-art methods, achieving significant improvements in reconstruction completeness, boundary consistency, and rendering fidelity.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recently, reconstructing scenes from a single panoramic image using advanced 3D Gaussian Splatting (3DGS) techniques has attracted growing interest. Panoramic images offer a 360$ imes$ 180 field of view (FoV), capturing the entire scene in a single shot. However, panoramic images introduce severe distortion, making it challenging to render 3D Gaussians into 2D distorted equirectangular space directly. Converting equirectangular images to cubemap projections partially alleviates this problem but introduces new challenges, such as projection distortion and discontinuities across cube-face boundaries. To address these limitations, we present a novel framework, named TPGS, to bridge continuous panoramic 3D scene reconstruction with perspective Gaussian splatting. Firstly, we introduce a Transition Plane between adjacent cube faces to enable smoother transitions in splatting directions and mitigate optimization ambiguity in the boundary region. Moreover, an intra-to-inter face optimization strategy is proposed to enhance local details and restore visual consistency across cube-face boundaries. Specifically, we optimize 3D Gaussians within individual cube faces and then fine-tune them in the stitched panoramic space. Additionally, we introduce a spherical sampling technique to eliminate visible stitching seams. Extensive experiments on indoor and outdoor, egocentric, and roaming benchmark datasets demonstrate that our approach outperforms existing state-of-the-art methods. Code and models will be available at https://github.com/zhijieshen-bjtu/TPGS.

Problem

• Address distortion in panoramic 3D Gaussian Splatting reconstruction
• Mitigate discontinuities across cubemap projection boundaries
• Enhance visual consistency in panoramic 3D scene rendering

Innovation

• Introduces Transition Plane for smoother splatting transitions
• Uses intra-to-inter face optimization for consistency
• Applies spherical sampling to eliminate stitching seams