Laplacian Score Sharpening for Mitigating Hallucination in Diffusion Models

2025-11-10

AI Summary

Diffusion models suffer from hallucination—generating incoherent or unrealistic samples—due to mode interpolation and score smoothing, yet existing methods lack real-time hallucination suppression during sampling. This work proposes a novel inference-time post-processing technique for the score function. We introduce, for the first time, a Laplacian-based (sharpness-aware) correction mechanism, leveraging the intrinsic relationship between the Laplacian operator and score uncertainty to suppress hallucinatory generation. To enable efficient computation of high-dimensional Laplacians, we design a variant of the Hutchinson trace estimator based on finite differences. Extensive experiments on 1D/2D synthetic distributions and high-dimensional image datasets demonstrate that our method significantly reduces hallucination rates, confirming both its effectiveness and cross-dimensional generalizability. This work establishes a new paradigm for enhancing the reliability of unconditional diffusion models.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Diffusion models, though successful, are known to suffer from hallucinations that create incoherent or unrealistic samples. Recent works have attributed this to the phenomenon of mode interpolation and score smoothening, but they lack a method to prevent their generation during sampling. In this paper, we propose a post-hoc adjustment to the score function during inference that leverages the Laplacian (or sharpness) of the score to reduce mode interpolation hallucination in unconditional diffusion models across 1D, 2D, and high-dimensional image data. We derive an efficient Laplacian approximation for higher dimensions using a finite-difference variant of the Hutchinson trace estimator. We show that this correction significantly reduces the rate of hallucinated samples across toy 1D/2D distributions and a high-dimensional image dataset. Furthermore, our analysis explores the relationship between the Laplacian and uncertainty in the score.

Problem

• Mitigating hallucination issues in diffusion models during sampling
• Reducing mode interpolation artifacts across 1D/2D and image data
• Addressing score smoothening problems through Laplacian-based correction

Innovation

• Post-hoc adjustment to score function during inference
• Leverages Laplacian sharpness to reduce hallucinations
• Uses efficient Laplacian approximation via Hutchinson estimator