DR-GS: Physically-Based Deformable and Relightable 2D Gaussians

2026-06-28

AI Summary

Existing deformable Gaussian splatting methods bake lighting into textures, leading to physically inconsistent appearance under dynamic deformation or changing illumination and precluding post-reconstruction material editing. To address this, this work proposes DR-GS, a novel framework that explicitly decouples geometry, lighting, and material representations within Gaussian splatting for the first time. By integrating physics-based inverse rendering, differentiable Gaussian splatting, and deformation-aware material modeling, DR-GS enables physically consistent relighting and parametric material editing. The method achieves state-of-the-art visual quality across static reconstruction, dynamic deformation, and relighting tasks, accurately preserving specular reflections and highlights on glossy surfaces, and supports end-to-end creation and high-quality post-editing of 3D assets.

Abstract

Gaussian splatting (GS) has garnered significant attention in VR/AR and digital content creation due to its explicit parameterization and efficient rendering capabilities. However, existing GS-based methods for deformable objects face two key limitations: (i) illumination is erroneously baked into textures, causing physically inconsistent responses under dynamic deformations and lighting changes; (ii) snapshot-based reconstruction restricts post-reconstruction material editing. To address these challenges, we propose Deformable and Relightable GS (DR-GS), a unified Gaussian framework that integrates physically-based inverse rendering, relighting, and deformation-aware manipulation. Through explicitly disentangling geometry, illumination, and material representations, DR-GS overcomes the limitations of static snapshots, resolving unrealistic appearance under varying conditions while enabling post-reconstruction parameter editing. Extensive experiments show that DR-GS achieves leading visual quality across static reconstruction, dynamic deformation, and relighting, reliably preserving reflections and specular highlights on glossy surfaces. It further establishes a fully decoupled geometry-illumination-material pipeline, enabling high-quality 3D asset creation and comprehensive post-editing.

Problem

• Gaussian splatting
• deformable objects
• relighting
• physically-based rendering
• material editing

Innovation

• Gaussian splatting
• physically-based rendering
• relighting
• deformable objects
• inverse rendering