GOGS: High-Fidelity Geometry and Relighting for Glossy Objects via Gaussian Surfels

2025-08-20

AI Summary

To address geometry-material coupling, multi-view inconsistency, and relighting artifacts in RGB inverse rendering of specular objects, this paper proposes a two-stage framework based on 2D Gaussian primitives. In Stage I, we integrate geometric priors from foundation models with differentiable 2D Gaussian ray tracing to model indirect illumination and enhance surface reconstruction robustness. In Stage II, spherical mipmap-based directional encoding is introduced to explicitly represent anisotropic specular reflection. Our method incorporates Gaussian splitting, Monte Carlo importance sampling, and physics-based rendering constraints to effectively suppress high-frequency noise and structural artifacts while preserving computational efficiency. Experiments demonstrate state-of-the-art performance in geometric reconstruction accuracy, material disentanglement quality, and photorealistic novel-view relighting—particularly for glossy objects under complex lighting conditions.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Inverse rendering of glossy objects from RGB imagery remains fundamentally limited by inherent ambiguity. Although NeRF-based methods achieve high-fidelity reconstruction via dense-ray sampling, their computational cost is prohibitive. Recent 3D Gaussian Splatting achieves high reconstruction efficiency but exhibits limitations under specular reflections. Multi-view inconsistencies introduce high-frequency surface noise and structural artifacts, while simplified rendering equations obscure material properties, leading to implausible relighting results. To address these issues, we propose GOGS, a novel two-stage framework based on 2D Gaussian surfels. First, we establish robust surface reconstruction through physics-based rendering with split-sum approximation, enhanced by geometric priors from foundation models. Second, we perform material decomposition by leveraging Monte Carlo importance sampling of the full rendering equation, modeling indirect illumination via differentiable 2D Gaussian ray tracing and refining high-frequency specular details through spherical mipmap-based directional encoding that captures anisotropic highlights. Extensive experiments demonstrate state-of-the-art performance in geometry reconstruction, material separation, and photorealistic relighting under novel illuminations, outperforming existing inverse rendering approaches.

Problem

• Inverse rendering of glossy objects suffers from inherent ambiguity and high computational cost
• 3D Gaussian Splatting exhibits limitations under specular reflections causing surface artifacts
• Simplified rendering equations obscure material properties leading to implausible relighting results

Innovation

• Two-stage framework using Gaussian surfels
• Physics-based rendering with split-sum approximation
• Differentiable Gaussian ray tracing for illumination