PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling

2026-06-24

AI Summary

This work addresses the high computational cost of existing diffusion-based single-image 3D reconstruction methods, which rely on iterative sampling. The authors propose PRISM, a novel framework that achieves high-quality, feed-forward 3D reconstruction without diffusion sampling by integrating geometric forward-warping priors with learnable residual correction for the first time. Through a two-stage training strategy—comprising latent-space supervised distillation followed by perceptual fine-tuning—the method effectively generalizes from synthetic to real-world scenes. Evaluated on three standard benchmarks, PRISM matches the reconstruction quality of state-of-the-art diffusion-based approaches while reducing per-scene inference time to just 36 seconds.

Abstract

Reconstructing 3D scenes from a single image is a fundamental challenge in computer vision, with broad applications in virtual reality, robotics, and content creation. Recent methods achieve outstanding performance by leveraging camera-controlled video diffusion models, but rely on iterative diffusion sampling, which greatly limits their practical deployment. We observe that geometric forward warping alone can cover the majority of a target view directly from the input image, with only a compact residual left for the encoder to correct. Motivated by this observation, we propose PRISM, a feed-forward framework that decomposes multi-view latent prediction into a parameter-free geometric prior and a learned residual correction, with no diffusion sampling required at inference. To enable generalization from purely synthetic training data, we devise a two-stage training strategy combining latents supervised distillation for geometric generalization and perceptual fine-tuning for appearance quality optimization. Extensive experiments on three benchmarks demonstrate that PRISM achieves competitive reconstruction quality compared with diffusion-based methods, while reducing inference time dramatically to only 36 seconds per scene.

Problem

• single-image 3D reconstruction
• feed-forward inference
• diffusion models
• 3D scene reconstruction
• computer vision

Innovation

• feed-forward reconstruction
• geometric warp-residual modeling
• single-image 3D reconstruction
• diffusion-free inference
• two-stage training