Seam360GS: Seamless 360° Gaussian Splatting from Real-World Omnidirectional Images

2025-08-27

AI Summary

Consumer-grade dual-fisheye cameras suffer from stitching discontinuities, parallax, and photometric inconsistencies in 360° panoramic image generation due to physical lens separation and severe angular distortion. Method: We propose the first 3D Gaussian splatting framework embedded with a differentiable dual-fisheye camera model. It jointly optimizes camera calibration parameters—including radial distortion coefficients and baseline—and 3D Gaussian distributions to explicitly model and correct both nonlinear optical distortions and physical inter-lens gaps. A novel panoramic photometric consistency loss is introduced to enforce end-to-end seamless 360° synthesis. Results: Extensive evaluation on multiple real-world dual-fisheye datasets demonstrates that our method significantly suppresses stitching artifacts, producing high-fidelity panoramas with visual continuity and geometric accuracy—outperforming state-of-the-art 360° rendering approaches.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Data transparency and provenance
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

360-degree visual content is widely shared on platforms such as YouTube and plays a central role in virtual reality, robotics, and autonomous navigation. However, consumer-grade dual-fisheye systems consistently yield imperfect panoramas due to inherent lens separation and angular distortions. In this work, we introduce a novel calibration framework that incorporates a dual-fisheye camera model into the 3D Gaussian splatting pipeline. Our approach not only simulates the realistic visual artifacts produced by dual-fisheye cameras but also enables the synthesis of seamlessly rendered 360-degree images. By jointly optimizing 3D Gaussian parameters alongside calibration variables that emulate lens gaps and angular distortions, our framework transforms imperfect omnidirectional inputs into flawless novel view synthesis. Extensive evaluations on real-world datasets confirm that our method produces seamless renderings-even from imperfect images-and outperforms existing 360-degree rendering models.

Problem

• Addressing imperfect 360° panoramas from dual-fisheye cameras
• Correcting lens separation and angular distortion artifacts
• Enabling seamless novel view synthesis from flawed inputs

Innovation

• Dual-fisheye camera model integrated into Gaussian splatting
• Joint optimization of 3D Gaussian parameters with calibration variables
• Simulates lens artifacts to create seamless 360-degree renderings