Problem
Research questions and friction points this paper is trying to address.
Reflection-aware
novel view synthesis
mirror scenes
multi-view diffusion models
Reflection-aware Generative Novel View Synthesis
🤖 AI Summary
提出Ref-GeNVS，一种无需训练的反射感知方法，通过将镜像视为互补视图并利用虚拟视图设置来解决多视角扩散模型在镜面场景中无法识别镜子的问题。
📝 Abstract
We propose Ref-GeNVS, a training-free, reflection-aware method for generative novel view synthesis (NVS) in mirror scenes. Existing multi-view diffusion models often fail to recognize the mirror in the scene and cannot exploit reflected content for scene generation. To fix this issue without additional training, our key idea is to treat a mirror image as two complementary views. From input images, we estimate the mirror plane and reflect camera poses to form virtual views. Based on this virtual view setup, we propose a two-stage generation method consisting of Mirror-gated attention and Reflection injection, which enables reflection-consistent NVS by explicitly leveraging reflection relationships in a multi-view diffusion model. Ref-GeNVS inherits the strong generalizability of the multi-view diffusion backbone, while it does not require finetuning. On synthetic and real scenes including mirrors, Ref-GeNVS outperforms recent generative NVS methods by generating reflection-consistent and contextually coherent novel views, revealing scene structure visible only through mirrors. Project page: https://kim-geonu.github.io/Ref-GeNVS/
Innovation
Methods, ideas, or system contributions that make the work stand out.
reflection-aware
novel view synthesis
mirror-gated attention
reflection injection
multi-view diffusion model
🔎 Similar Papers
No similar papers found.
