DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusion

2025-07-30

AI Summary

This work addresses geometric distortion and insufficient instance-level detail in single-view 3D scene reconstruction. We propose DepR, a depth-guided, instance-level diffusion reconstruction framework. Methodologically, DepR is the first to integrate depth maps as a strong geometric prior throughout both training and inference: (i) a depth-conditioned encoding module fuses multi-scale depth features; (ii) DDIM sampling enables high-fidelity, depth-constrained instance generation; and (iii) a layout optimization strategy progressively composes instances into a coherent 3D scene. Extensive experiments on synthetic and real-world datasets demonstrate that DepR achieves state-of-the-art performance in geometric accuracy, instance completeness, and cross-domain generalization—significantly outperforming existing single-view 3D reconstruction methods.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We propose DepR, a depth-guided single-view scene reconstruction framework that integrates instance-level diffusion within a compositional paradigm. Instead of reconstructing the entire scene holistically, DepR generates individual objects and subsequently composes them into a coherent 3D layout. Unlike previous methods that use depth solely for object layout estimation during inference and therefore fail to fully exploit its rich geometric information, DepR leverages depth throughout both training and inference. Specifically, we introduce depth-guided conditioning to effectively encode shape priors into diffusion models. During inference, depth further guides DDIM sampling and layout optimization, enhancing alignment between the reconstruction and the input image. Despite being trained on limited synthetic data, DepR achieves state-of-the-art performance and demonstrates strong generalization in single-view scene reconstruction, as shown through evaluations on both synthetic and real-world datasets.

Problem

• Reconstructs 3D scenes from single-view images using depth guidance
• Integrates instance-level diffusion for individual object generation
• Enhances alignment between reconstruction and input via depth conditioning

Innovation

• Depth-guided conditioning for shape priors
• DDIM sampling with depth guidance
• Instance-level diffusion in compositional paradigm