EGGS: Exchangeable 2D/3D Gaussian Splatting for Geometry-Appearance Balanced Novel View Synthesis

2025-12-02

AI Summary

Novel view synthesis (NVS) faces a fundamental trade-off between geometric inconsistency in 3D Gaussian splatting (3DGS) and texture distortion in 2D Gaussian splatting (2DGS). To address this, we propose EGGS—a novel exchangeable 2D/3D hybrid Gaussian representation framework. EGGS unifies rendering via hybrid Gaussian rasterization, enables adaptive switching between 2D and 3D Gaussians during training, and introduces frequency-domain decoupled optimization to jointly enhance multi-view consistency and geometric fidelity. Implemented with CUDA acceleration, it achieves the first end-to-end joint optimization of both 2D and 3D Gaussian representations. On multiple standard benchmarks, EGGS outperforms state-of-the-art methods: average PSNR improves by 1.8 dB, depth error reduces by 22%, and inference speed exceeds 30 FPS—delivering high-fidelity rendering, accurate geometry reconstruction, and real-time computational efficiency.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Search and Optimization: Mixed Discrete/Continuous Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Novel view synthesis (NVS) is crucial in computer vision and graphics, with wide applications in AR, VR, and autonomous driving. While 3D Gaussian Splatting (3DGS) enables real-time rendering with high appearance fidelity, it suffers from multi-view inconsistencies, limiting geometric accuracy. In contrast, 2D Gaussian Splatting (2DGS) enforces multi-view consistency but compromises texture details. To address these limitations, we propose Exchangeable Gaussian Splatting (EGGS), a hybrid representation that integrates 2D and 3D Gaussians to balance appearance and geometry. To achieve this, we introduce Hybrid Gaussian Rasterization for unified rendering, Adaptive Type Exchange for dynamic adaptation between 2D and 3D Gaussians, and Frequency-Decoupled Optimization that effectively exploits the strengths of each type of Gaussian representation. Our CUDA-accelerated implementation ensures efficient training and inference. Extensive experiments demonstrate that EGGS outperforms existing methods in rendering quality, geometric accuracy, and efficiency, providing a practical solution for high-quality NVS.

Problem

• Balances appearance quality and geometric accuracy in novel view synthesis
• Resolves multi-view inconsistency in 3D Gaussian Splatting methods
• Addresses texture detail loss in 2D Gaussian Splatting approaches

Innovation

• Hybrid Gaussian Rasterization for unified rendering
• Adaptive Type Exchange for dynamic 2D/3D adaptation
• Frequency-Decoupled Optimization to exploit Gaussian strengths