Generative Myopia: Why Diffusion Models Fail at Structure

2025-11-23

AI Summary

Diffusion models suffer from “generative myopia” in structural graph generation: they over-optimize statistical likelihood while neglecting spectrally critical sparse substructures—such as rare bridge edges essential for connectivity—leading to catastrophic failure in combinatorial tasks like graph sparsification. To address this, we propose Spectral-Weighted Diffusion, a novel framework that incorporates spectral priors into the training objective via effective resistance-based reweighting of the variational loss, with zero inference-time overhead. This is the first method to explicitly embed graph spectral information during training, thereby alleviating gradient starvation and smoothing the optimization landscape. Theoretical analysis and extensive experiments demonstrate that our approach achieves 100% connectivity on adversarial sparsification benchmarks—matching the performance of an optimal spectral oracle—while standard diffusion models completely fail (0% connectivity).

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Content-based information diffusion
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Graph Diffusion Models (GDMs) optimize for statistical likelihood, implicitly acting as extbf{frequency filters} that favor abundant substructures over spectrally critical ones. We term this phenomenon extbf{Generative Myopia}. In combinatorial tasks like graph sparsification, this leads to the catastrophic removal of ``rare bridges,'' edges that are structurally mandatory ($R_{ ext{eff}} approx 1$) but statistically scarce. We prove theoretically and empirically that this failure is driven by extbf{Gradient Starvation}: the optimization landscape itself suppresses rare structural signals, rendering them unlearnable regardless of model capacity. To resolve this, we introduce extbf{Spectrally-Weighted Diffusion}, which re-aligns the variational objective using Effective Resistance. We demonstrate that spectral priors can be amortized into the training phase with zero inference overhead. Our method eliminates myopia, matching the performance of an optimal Spectral Oracle and achieving extbf{100% connectivity} on adversarial benchmarks where standard diffusion fails completely (0%).

Problem

• Diffusion models prioritize frequent structures over critical rare ones
• This causes catastrophic failure in preserving structurally essential graph bridges
• Gradient starvation suppresses rare structural signals during model optimization

Innovation

• Spectrally-Weighted Diffusion using Effective Resistance
• Amortized spectral priors with zero inference overhead
• Re-aligned variational objective eliminating generative myopia