UFV-Splatter: Pose-Free Feed-Forward 3D Gaussian Splatting Adapted to Unfavorable Views

2025-07-29

AI Summary

Pre-trained pose-free feedforward 3D Gaussian Splatting (3DGS) models suffer significant degradation in novel-view synthesis quality under non-ideal input viewpoints. To address this, we propose the first pose-agnostic feedforward adaptive framework. Our method introduces: (1) a lightweight Gaussian adapter module leveraging Low-Rank Adaptation (LoRA) for efficient fine-tuning of pre-trained 3DGS; (2) a Gaussian distribution re-centering and geometric alignment rendering strategy to enhance cross-view geometric consistency; and (3) training exclusively on ideal-view data, eliminating the need for ground-truth annotations from non-ideal viewpoints. Experiments on Google Scanned Objects and OmniObject3D demonstrate substantial improvements in rendering quality under non-ideal viewpoints, with PSNR and SSIM approaching those achieved under ideal viewpoints. Our approach establishes a new paradigm for robust deployment of 3DGS in real-world scenarios.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Agentic search

Abstract

This paper presents a pose-free, feed-forward 3D Gaussian Splatting (3DGS) framework designed to handle unfavorable input views. A common rendering setup for training feed-forward approaches places a 3D object at the world origin and renders it from cameras pointed toward the origin -- i.e., from favorable views, limiting the applicability of these models to real-world scenarios involving varying and unknown camera poses. To overcome this limitation, we introduce a novel adaptation framework that enables pretrained pose-free feed-forward 3DGS models to handle unfavorable views. We leverage priors learned from favorable images by feeding recentered images into a pretrained model augmented with low-rank adaptation (LoRA) layers. We further propose a Gaussian adapter module to enhance the geometric consistency of the Gaussians derived from the recentered inputs, along with a Gaussian alignment method to render accurate target views for training. Additionally, we introduce a new training strategy that utilizes an off-the-shelf dataset composed solely of favorable images. Experimental results on both synthetic images from the Google Scanned Objects dataset and real images from the OmniObject3D dataset validate the effectiveness of our method in handling unfavorable input views.

Problem

• Adapts 3D Gaussian Splatting for unfavorable camera views
• Enhances geometric consistency with Gaussian adapter module
• Uses favorable image datasets for training adaptation

Innovation

• Pose-free feed-forward 3D Gaussian Splatting adaptation
• Low-rank adaptation (LoRA) for pretrained models
• Gaussian adapter module enhances geometric consistency