Personalize Your Gaussian: Consistent 3D Scene Personalization from a Single Image

2025-05-20

AI Summary

To address the core challenges of multi-view inconsistency and poor reference consistency in single-reference-image-driven 3D scene personalization, this paper proposes a progressive single-view appearance propagation framework. Our method integrates the 3D generation priors of pre-trained diffusion models with geometry-guided LoRA-based efficient fine-tuning, enabling cross-view appearance consistency modeling directly on 3D Gaussian Splatting (3DGS) representations and effectively mitigating viewpoint bias induced by single-view input. Crucially, we decouple appearance propagation into coarse-grained geometric constraint enforcement and fine-grained texture refinement via iterative optimization, significantly improving both multi-view consistency and image fidelity. Extensive experiments demonstrate that our approach consistently outperforms existing single-image 3D personalization methods on real-world scenes. This work establishes a new paradigm for few-shot 3D content creation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Personalizing 3D scenes from a single reference image enables intuitive user-guided editing, which requires achieving both multi-view consistency across perspectives and referential consistency with the input image. However, these goals are particularly challenging due to the viewpoint bias caused by the limited perspective provided in a single image. Lacking the mechanisms to effectively expand reference information beyond the original view, existing methods of image-conditioned 3DGS personalization often suffer from this viewpoint bias and struggle to produce consistent results. Therefore, in this paper, we present Consistent Personalization for 3D Gaussian Splatting (CP-GS), a framework that progressively propagates the single-view reference appearance to novel perspectives. In particular, CP-GS integrates pre-trained image-to-3D generation and iterative LoRA fine-tuning to extract and extend the reference appearance, and finally produces faithful multi-view guidance images and the personalized 3DGS outputs through a view-consistent generation process guided by geometric cues. Extensive experiments on real-world scenes show that our CP-GS effectively mitigates the viewpoint bias, achieving high-quality personalization that significantly outperforms existing methods. The code will be released at https://github.com/Yuxuan-W/CP-GS.

Problem

• Achieving multi-view consistency from single image
• Overcoming viewpoint bias in 3D scene personalization
• Propagating reference appearance to novel perspectives

Innovation

• Propagates single-view reference appearance progressively
• Integrates pre-trained image-to-3D generation and LoRA fine-tuning
• Uses view-consistent generation guided by geometric cues