Perspective-aware 3D Gaussian Inpainting with Multi-view Consistency

2025-10-13

AI Summary

Addressing the critical challenge of ensuring multi-view consistency in 3D Gaussian inpainting, this paper proposes PAInpainter—a high-fidelity 3D scene inpainting method tailored for VR and multimedia applications. Its core innovation lies in constructing a view graph to enable adaptive multi-view sampling, synergistically integrating inter-view prior content propagation with neighboring-view consistency verification. This joint optimization harmonizes diffusion priors, 3D Gaussian representations, and graph-structured sampling. Evaluated on SPIn-NeRF and NeRFiller benchmarks, PAInpainter achieves PSNR scores of 26.03 dB and 29.51 dB, respectively—substantially outperforming state-of-the-art methods. The approach delivers superior texture fidelity, global geometric consistency across views, and strong generalization across diverse scenes, demonstrating robustness for real-world immersive applications.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

3D Gaussian inpainting, a critical technique for numerous applications in virtual reality and multimedia, has made significant progress with pretrained diffusion models. However, ensuring multi-view consistency, an essential requirement for high-quality inpainting, remains a key challenge. In this work, we present PAInpainter, a novel approach designed to advance 3D Gaussian inpainting by leveraging perspective-aware content propagation and consistency verification across multi-view inpainted images. Our method iteratively refines inpainting and optimizes the 3D Gaussian representation with multiple views adaptively sampled from a perspective graph. By propagating inpainted images as prior information and verifying consistency across neighboring views, PAInpainter substantially enhances global consistency and texture fidelity in restored 3D scenes. Extensive experiments demonstrate the superiority of PAInpainter over existing methods. Our approach achieves superior 3D inpainting quality, with PSNR scores of 26.03 dB and 29.51 dB on the SPIn-NeRF and NeRFiller datasets, respectively, highlighting its effectiveness and generalization capability.

Problem

• Ensuring multi-view consistency in 3D Gaussian inpainting
• Enhancing global consistency and texture fidelity restoration
• Advancing 3D scene reconstruction with perspective-aware propagation

Innovation

• Leverages perspective-aware content propagation for inpainting
• Iteratively refines inpainting with adaptive multi-view sampling
• Verifies consistency across neighboring views for fidelity