HAD: Hallucination-Aware Diffusion Priors for 3D Reconstruction

2026-05-16

AI Summary

In sparse-view 3D reconstruction, diffusion priors often introduce hallucinated artifacts inconsistent with the input views, compromising reconstruction fidelity. This work proposes a hallucination-aware mechanism that leverages a pretrained novel view synthesis network to estimate pixel-level hallucination scores for diffusion-augmented images and selectively masks unreliable regions during 3D reconstruction. To further enhance geometric consistency, the method fuses multi-view conditionally generated augmented images. By explicitly integrating hallucination detection and suppression into the diffusion prior–guided reconstruction pipeline— a first in the field—this approach significantly reduces artifacts and achieves state-of-the-art performance across multiple novel view synthesis benchmarks.

Abstract

Diffusion priors have recently demonstrated strong capability in enhancing the quality of sparse-view 3D reconstruction by augmenting training views at novel viewpoints, but they inevitably introduce hallucinated content -- artifacts inconsistent with the input views -- into the final 3D model. To address this challenge, we propose Hallucination-Aware Diffusion prior (HAD), which estimates pixel-wise hallucination score maps for augmented images by leveraging multi-view reasoning capabilities from a feedforward novel view synthesis (NVS) network pre-trained on large-scale 3D data. These hallucination scores enable selective masking of unreliable pixels during the progressive 3D reconstruction procedure, preventing the introduction of non-existent artifacts into the 3D model. To further enhance performance, we create multiple versions of augmented images at each novel view by conditioning the diffusion prior on different input views, which are then fused into a final image that leverages the broader context across all input views. We show that our method substantially reduces hallucination artifacts in diffusion-assisted 3D reconstruction, thereby achieving state-of-the-art performance across multiple benchmarks on novel view synthesis. Our project are publicly available at \href{https://xiliu8006.github.io/HAD-Project-website/}{project website}.

Problem

• hallucination
• 3D reconstruction
• diffusion priors
• sparse-view
• novel view synthesis

Innovation

• Hallucination-Aware
• Diffusion Prior
• 3D Reconstruction
• Novel View Synthesis
• Multi-view Reasoning