Problem
Research questions and friction points this paper is trying to address.
Single-view 3D reconstruction
diffusion models
color deviation
view inconsistency
high-frequency details
Single Image to Textured 3D Object Generation in Frequency Domain: From Theory to Pipeline
🤖 AI Summary
本文针对单视图3D重建中信息缺失、颜色偏差及视角不一致问题，提出了一种基于频域的多扩散先验混合优化框架Morpheus3D，有效提升了3D对象生成质量。
📝 Abstract
Single-view 3D reconstruction, also known as image-to-3D, is a persistently challenging task due to the extreme lack of information. Recently, diffusion models pre-trained on large-scale datasets served as 2D priors are used to solve the ill-posed task but suffer from color deviation and view inconsistency, which can be curbed by using diffusion models fine-tuned with 3D annotated data served as 3D priors. However, 3D priors lack high-frequency details, which cannot be solved by direct complementation with 2D priors in spatial domain for introducing erroneous low-frequency 2D prior guidance. In this paper, we revisit the characteristics of different diffusion priors from the frequency perspective. Based on our observations, we theoretically present a unified framework of hybrid optimization using multiple diffusion priors in frequency domain. Under this framework, we further propose Morpheus3D, a pipeline of 3D object generation from any single unposed image in the wild. Morpheus3D enhances 3D prior with high-pass image-prompt 2D prior guidance to reconstruct high-quality 3D objects while effectively suppressing view inconsistency, low-frequency color deviation, and high-frequency lacking problems. Both quantitative and qualitative experiments on the public and our collected datasets with complex textures show that our method exhibits significant improvements in generation quality.
Innovation
Methods, ideas, or system contributions that make the work stand out.
frequency domain
diffusion priors
hybrid optimization
high-pass image-prompt 2D prior
Morpheus3D
🔎 Similar Papers
No similar papers found.
