Model-free filtering in high dimensions via projection and score-based diffusions

2025-10-27

AI Summary

This work addresses the problem of recovering a latent variable $X$, supported on an unknown low-dimensional manifold $mathcal{M}$, from high-dimensional noisy observations $Y = X + varepsilon$, where $X sim mathbb{P}^X$ is accessible only via i.i.d. samples. We propose a distribution-free manifold denoising filter that integrates score matching with diffusion models to estimate the metric projection operator onto $mathcal{M}$. A semiparametric observation model is constructed using isotropic Laplacian noise, and posterior concentration is rigorously established in the high-dimensional limit. Theoretically, the proposed estimator approximates the true metric projection with high probability. Empirically, it achieves substantial improvements over state-of-the-art manifold denoising methods on both synthetic and real-world datasets.

Technology Category

• Machine Learning: Learning with Manifolds
• Intelligent Robots: State Estimation
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

We consider the problem of recovering a latent signal $X$ from its noisy observation $Y$. The unknown law $mathbb{P}^X$ of $X$, and in particular its support $mathscr{M}$, are accessible only through a large sample of i.i.d. observations. We further assume $mathscr{M}$ to be a low-dimensional submanifold of a high-dimensional Euclidean space $mathbb{R}^d$. As a filter or denoiser $widehat X$, we suggest an estimator of the metric projection $π_{mathscr{M}}(Y)$ of $Y$ onto the manifold $mathscr{M}$. To compute this estimator, we study an auxiliary semiparametric model in which $Y$ is obtained by adding isotropic Laplace noise to $X$. Using score matching within a corresponding diffusion model, we obtain an estimator of the Bayesian posterior $mathbb{P}^{X mid Y}$ in this setup. Our main theoretical results show that, in the limit of high dimension $d$, this posterior $mathbb{P}^{Xmid Y}$ is concentrated near the desired metric projection $π_{mathscr{M}}(Y)$.

Problem

• Recovering latent signals from noisy observations in high dimensions
• Estimating metric projections onto low-dimensional manifolds
• Concentrating Bayesian posteriors near manifold projections via diffusion

Innovation

• Projecting noisy observations onto low-dimensional manifold
• Using score matching for Bayesian posterior estimation
• Applying diffusion models to recover latent signals