Provable Maximum Entropy Manifold Exploration via Diffusion Models

2025-06-18

AI Summary

For decision-making problems requiring genuinely novel designs—such as scientific discovery—this paper proposes a generative exploration framework that does not rely on explicit uncertainty estimation. The method leverages the implicit data manifold encoded by pretrained diffusion models, formalizing exploration as maximum-entropy density learning constrained to this manifold. Theoretically, we establish, for the first time, an analytical relationship between the entropy of the diffusion-induced density and its score function. Algorithmically, we design a sequential fine-tuning procedure based on mirror descent, ensuring both convergence guarantees and scalability. Empirical evaluation on synthetic benchmarks and high-dimensional text-to-image generation tasks demonstrates that our approach significantly improves sample novelty and diversity while provably converging to the optimal exploration solution.

Technology Category

• Search and Optimization: Learning to Search
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning with Manifolds

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

Exploration is critical for solving real-world decision-making problems such as scientific discovery, where the objective is to generate truly novel designs rather than mimic existing data distributions. In this work, we address the challenge of leveraging the representational power of generative models for exploration without relying on explicit uncertainty quantification. We introduce a novel framework that casts exploration as entropy maximization over the approximate data manifold implicitly defined by a pre-trained diffusion model. Then, we present a novel principle for exploration based on density estimation, a problem well-known to be challenging in practice. To overcome this issue and render this method truly scalable, we leverage a fundamental connection between the entropy of the density induced by a diffusion model and its score function. Building on this, we develop an algorithm based on mirror descent that solves the exploration problem as sequential fine-tuning of a pre-trained diffusion model. We prove its convergence to the optimal exploratory diffusion model under realistic assumptions by leveraging recent understanding of mirror flows. Finally, we empirically evaluate our approach on both synthetic and high-dimensional text-to-image diffusion, demonstrating promising results.

Problem

• Leveraging generative models for exploration without uncertainty quantification
• Maximizing entropy over data manifold from pre-trained diffusion models
• Developing scalable exploration via diffusion model score function connection

Innovation

• Maximizes entropy over diffusion model manifold
• Uses score function for scalable density estimation
• Employs mirror descent for sequential fine-tuning