Efficiently Access Diffusion Fisher: Within the Outer Product Span Space

2025-05-29

AI Summary

Diffusion Fisher information (DF) is computationally expensive and challenging to estimate accurately in diffusion models; existing automatic-differentiation-based approximations lack theoretical guarantees and incur high computational overhead. Method: We establish that DF lies strictly within a low-dimensional subspace spanned by the outer product of the score estimator and the initial data, enabling a novel trace and matrix-vector product approximation that avoids both automatic differentiation and Hessian computation. We derive rigorous theoretical error bounds for this approximation and conduct the first numerical verification that the PF-ODE mapping satisfies the Wasserstein gradient flow property. Results: Experiments demonstrate that our method reduces relative error by a factor of 3.2 and cuts computation time by 72% compared to baselines, significantly improving both accuracy and efficiency in likelihood evaluation and adjoint optimization tasks.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning with Manifolds
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent Diffusion models (DMs) advancements have explored incorporating the second-order diffusion Fisher information (DF), defined as the negative Hessian of log density, into various downstream tasks and theoretical analysis. However, current practices typically approximate the diffusion Fisher by applying auto-differentiation to the learned score network. This black-box method, though straightforward, lacks any accuracy guarantee and is time-consuming. In this paper, we show that the diffusion Fisher actually resides within a space spanned by the outer products of score and initial data. Based on the outer-product structure, we develop two efficient approximation algorithms to access the trace and matrix-vector multiplication of DF, respectively. These algorithms bypass the auto-differentiation operations with time-efficient vector-product calculations. Furthermore, we establish the approximation error bounds for the proposed algorithms. Experiments in likelihood evaluation and adjoint optimization demonstrate the superior accuracy and reduced computational cost of our proposed algorithms. Additionally, based on the novel outer-product formulation of DF, we design the first numerical verification experiment for the optimal transport property of the general PF-ODE deduced map.

Problem

• Efficiently approximating diffusion Fisher information without auto-differentiation
• Developing algorithms for trace and matrix-vector multiplication of DF
• Verifying optimal transport property of PF-ODE deduced map

Innovation

• DF resides in outer product space
• Efficient trace and matrix-vector algorithms
• Bypass auto-differentiation with vector-products