🤖 AI Summary
This work addresses the challenge that screen-space artifacts in real-world images—such as lens smudges and UI watermarks—are often erroneously reconstructed as floating 3D objects, severely degrading novel view synthesis quality. To resolve this, we propose the first unsupervised framework that jointly optimizes a 3D Gaussian point cloud and a learnable 2D overlay layer. By leveraging multi-view geometric consistency, our method automatically disentangles static artifacts from genuine 3D structure without requiring manual annotations or prior knowledge. Evaluated on both synthetic and real-world datasets, the approach achieves up to a 9 dB PSNR improvement over the original 3D Gaussian Splatting baseline, significantly enhancing reconstruction fidelity while accurately preserving artifact content.
📝 Abstract
Novel View Synthesis (NVS) methods, such as 3D Gaussian Splatting (3DGS), rely heavily on the assumption of clean, multi-view consistent, posed input images. Real-world captures can violate this assumption due to screen-space artifacts-static occlusions fixed to the 2D image plane rather than to the 3D world. Common examples include physical sensor defects, environmental obstructions (such as rain or mud on the lens enclosure), capture obstructions (such as a thumb over the camera sensor or a dashboard visible in dashcam footage), and digital overlays (such as watermarks or UI elements). When present, they are erroneously baked into the 3D geometry as "floaters" or near-camera artifacts, degrading the quality of novel-view rendering. In this work, we propose SSA-3DGS, an unsupervised framework that jointly optimizes a 3D scene and a learnable 2D overlay to recover a clean 3D scene and the corrupting artifacts. By exploiting geometric consensus across views, our method effectively disentangles static artifacts from the 3D scene geometry without supervision or manual input. Across diverse synthetic corruptions and a self-captured real-world dataset, SSA-3DGS improves reconstruction fidelity by up to 9 dB PSNR over 3DGS trained on the same corrupted inputs, while faithfully preserving the corrupting artifact.