RobustGS: Unified Boosting of Feedforward 3D Gaussian Splatting under Low-Quality Conditions

2025-08-05

AI Summary

Feed-forward 3D Gaussian Splatting (3DGS) reconstruction suffers from geometric distortion and quality degradation when trained on multi-view images corrupted by noise, low illumination, rain, fog, and other imaging degradations. Method: We propose a plug-and-play multi-view feature enhancement module that requires no modification to the original 3DGS training pipeline. Our approach introduces: (1) a generalized degradation learner for unified modeling of diverse image degradations; (2) a semantic-aware state-space model enabling degradation-aware representation learning and adaptive feature restoration in latent space; and (3) cross-view fine-grained semantic alignment to improve 3D geometric consistency. Contribution/Results: The method significantly improves robustness of feed-forward 3DGS under low-quality inputs. Extensive experiments demonstrate state-of-the-art reconstruction accuracy across multiple degradation scenarios, substantially outperforming existing feed-forward 3DGS methods.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Feature Construction/Reformulation
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Feedforward 3D Gaussian Splatting (3DGS) overcomes the limitations of optimization-based 3DGS by enabling fast and high-quality reconstruction without the need for per-scene optimization. However, existing feedforward approaches typically assume that input multi-view images are clean and high-quality. In real-world scenarios, images are often captured under challenging conditions such as noise, low light, or rain, resulting in inaccurate geometry and degraded 3D reconstruction. To address these challenges, we propose a general and efficient multi-view feature enhancement module, RobustGS, which substantially improves the robustness of feedforward 3DGS methods under various adverse imaging conditions, enabling high-quality 3D reconstruction. The RobustGS module can be seamlessly integrated into existing pretrained pipelines in a plug-and-play manner to enhance reconstruction robustness. Specifically, we introduce a novel component, Generalized Degradation Learner, designed to extract generic representations and distributions of multiple degradations from multi-view inputs, thereby enhancing degradation-awareness and improving the overall quality of 3D reconstruction. In addition, we propose a novel semantic-aware state-space model. It first leverages the extracted degradation representations to enhance corrupted inputs in the feature space. Then, it employs a semantic-aware strategy to aggregate semantically similar information across different views, enabling the extraction of fine-grained cross-view correspondences and further improving the quality of 3D representations. Extensive experiments demonstrate that our approach, when integrated into existing methods in a plug-and-play manner, consistently achieves state-of-the-art reconstruction quality across various types of degradations.

Problem

• Enhances 3D reconstruction under low-quality imaging conditions
• Improves robustness of feedforward 3D Gaussian Splatting methods
• Addresses noise, low light, and rain in multi-view images

Innovation

• Multi-view feature enhancement module RobustGS
• Generalized Degradation Learner for degradation-awareness
• Semantic-aware state-space model for cross-view aggregation