RefGlass-GS: A UAV-Enabled Fusion Framework for Photorealistic, Semantic and Interactive Digitization of Reflective Glass Facades via Gaussian Splatting

2026-06-27

AI Summary

This study addresses geometric distortion, view-dependent texture artifacts, and challenges in semantic enhancement inherent in the digital reconstruction of reflective glass facades. To this end, the authors propose RefGlass-GS, an end-to-end framework that enables photorealistic, semantically enriched, and interactive reconstruction. Key innovations include structure-prior-guided segmentation of glass panels, a drone trajectory planning strategy optimized for maximal reflection coverage, a Gaussian Splatting formulation enhanced with a Reflection MLP and tailored regularization terms, and an object-oriented organization of digital twin data. Experimental results demonstrate significant improvements: panel segmentation achieves a 0.1927 increase in mIoU and, for the first time, enables instance-level extraction; viewpoint planning yields a 13.15 dB gain in PSNR for novel view synthesis; and the overall modeling pipeline improves average PSNR by 5.08 dB in reflective scenes.

Abstract

Existing digitization of buildings with reflective glass facades suffers from geometric reconstruction distortion, unrealistic view-dependent texture rendering, and difficulties in object-based semantic enhancement. Therefore, we propose RefGlass-GS, a fusion framework that enables end-to-end UAV-based photorealistic, semantic, and interactive digitization of reflective glass facades. The contributions include: (1) proposing an individual glass panel segmentation method based on maximum a posteriori estimation with structural regularities, robust to severe reflection and background interference; (2) formulating a UAV viewpoint planning optimization function that maximizes the coverage of view-dependent appearance for sufficient data capture; (3) developing an optimized Gaussian Splatting framework with a Reflection MLP, a novel deferred shading function, and two enhanced regularization terms for effective modeling of high-frequency near-field reflections; (4) introducing a standardized data organization paradigm for structuring GS-based representations into object-based models, facilitating interactive facility management on digital twin platforms. Experiments on real-world reflective glass facade scenes validate the effectiveness and superiority of the proposed method. Specifically, the glass panel segmentation achieves an improvement of 0.1927 in mIoU over SOTA methods, and only our method enables instance-level panel extraction. The UAV view planning improves novel view synthesis for reflective facades by 13.15 dB in PSNR compared to commercially used nap-of-the-object planning methods. The RefGlass-GS modeling outperforms SOTA Gaussian Splatting approaches for reflective scenes with an average improvement of 5.08 dB in PSNR.

Problem

• reflective glass facades
• geometric reconstruction distortion
• view-dependent texture rendering
• semantic enhancement
• digitalization

Innovation

• Gaussian Splatting
• reflective glass facade
• UAV viewpoint planning
• semantic segmentation
• digital twin