CloseUpShot: Close-up Novel View Synthesis from Sparse-views via Point-conditioned Diffusion Model

2025-11-17

AI Summary

To address challenges—including detail loss, background leakage, and geometric inconsistency—in close-range 3D scene reconstruction and novel view synthesis from sparse input views, this paper proposes a hierarchical reconstruction framework based on point-cloud-conditioned diffusion models. Methodologically, we design an occlusion-aware noise suppression strategy and a global structural guidance mechanism, leveraging dense multi-view point clouds as geometric priors to constrain the diffusion process; we further introduce pixel-level mapping optimization and temporal point cloud fusion to ensure spatiotemporal consistency. Crucially, we explicitly embed geometric structure into the diffusion modeling, enabling fine-grained texture recovery and holistic structural integrity under sparse conditioning. Evaluated on multiple close-range datasets, our method achieves state-of-the-art performance in PSNR, LPIPS, and visual quality, demonstrating superior fidelity in both geometric coherence and textural detail.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Reconstructing 3D scenes and synthesizing novel views from sparse input views is a highly challenging task. Recent advances in video diffusion models have demonstrated strong temporal reasoning capabilities, making them a promising tool for enhancing reconstruction quality under sparse-view settings. However, existing approaches are primarily designed for modest viewpoint variations, which struggle in capturing fine-grained details in close-up scenarios since input information is severely limited. In this paper, we present a diffusion-based framework, called CloseUpShot, for close-up novel view synthesis from sparse inputs via point-conditioned video diffusion. Specifically, we observe that pixel-warping conditioning suffers from severe sparsity and background leakage in close-up settings. To address this, we propose hierarchical warping and occlusion-aware noise suppression, enhancing the quality and completeness of the conditioning images for the video diffusion model. Furthermore, we introduce global structure guidance, which leverages a dense fused point cloud to provide consistent geometric context to the diffusion process, to compensate for the lack of globally consistent 3D constraints in sparse conditioning inputs. Extensive experiments on multiple datasets demonstrate that our method outperforms existing approaches, especially in close-up novel view synthesis, clearly validating the effectiveness of our design.

Problem

• Synthesizing close-up novel views from sparse input images
• Addressing fine-grained detail loss in limited viewpoint scenarios
• Enhancing geometric consistency for sparse 3D reconstruction

Innovation

• Point-conditioned video diffusion for sparse views
• Hierarchical warping and occlusion-aware noise suppression
• Global structure guidance using dense fused point cloud