Manifold Aware Denoising Score Matching (MAD)

2026-03-02

AI Summary

Learning data distributions on manifolds typically requires jointly modeling both the manifold structure and the distribution, which incurs high computational costs. This work proposes an efficient score matching method that decomposes the score function into an analytically constructed basis term—implicitly encoding geometric priors of the manifold—and a residual term to be learned. By doing so, the model focuses solely on learning the distribution within the manifold, without explicitly modeling the manifold itself. Notably, this approach yields, for the first time, closed-form expressions for the basis score functions on complex manifolds such as rotation matrices and discrete distributions. Experiments demonstrate that the method substantially reduces computational burden across several canonical manifold distributions while maintaining high accuracy.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Other Foundations of Computer Vision
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

A major focus in designing methods for learning distributions defined on manifolds is to alleviate the need to implicitly learn the manifold so that learning can concentrate on the data distribution within the manifold. However, accomplishing this often leads to compute-intensive solutions. In this work, we propose a simple modification to denoising score-matching in the ambient space to implicitly account for the manifold, thereby reducing the burden of learning the manifold while maintaining computational efficiency. Specifically, we propose a simple decomposition of the score function into a known component $s^{base}$ and a remainder component $s-s^{base}$ (the learning target), with the former implicitly including information on where the data manifold resides. We derive known components $s^{base}$ in analytical form for several important cases, including distributions over rotation matrices and discrete distributions, and use them to demonstrate the utility of this approach in those cases.

Problem

• manifold learning
• denoising score matching
• computational efficiency
• data distribution
• score function

Innovation

• Manifold Aware Denoising Score Matching
• score function decomposition
• ambient space learning
• analytical manifold prior
• efficient distribution learning