Sparsely Supervised Diffusion

2026-02-02

AI Summary

This work addresses the issue of global structural inconsistency in diffusion models, which often arises from the locality of their denoising mechanisms. To mitigate this, the authors propose a sparsely supervised learning approach that masks up to 98% of pixels during training, retaining only critical contextual information to guide globally coherent generation. The method is efficiently implementable with minimal code modifications, fully compatible with standard denoising objectives, and substantially reduces pixel-level supervision while alleviating training instability on small datasets. Experimental results demonstrate that the proposed framework achieves competitive FID scores across multiple benchmarks, effectively suppresses overfitting and memorization, and significantly enhances the global consistency of generated images.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Diffusion models have shown remarkable success across a wide range of generative tasks. However, they often suffer from spatially inconsistent generation, arguably due to the inherent locality of their denoising mechanisms. This can yield samples that are locally plausible but globally inconsistent. To mitigate this issue, we propose sparsely supervised learning for diffusion models, a simple yet effective masking strategy that can be implemented with only a few lines of code. Interestingly, the experiments show that it is safe to mask up to 98\% of pixels during diffusion model training. Our method delivers competitive FID scores across experiments and, most importantly, avoids training instability on small datasets. Moreover, the masking strategy reduces memorization and promotes the use of essential contextual information during generation.

Problem

• diffusion models
• spatial inconsistency
• global coherence
• denoising mechanisms
• generative tasks

Innovation

• sparsely supervised learning
• diffusion models
• masking strategy
• spatial consistency
• contextual information