Precise Performance of Linear Denoisers in the Proportional Regime

2026-03-19

AI Summary

This work addresses the problem of constructing high-performance linear denoisers from a finite number of samples when the signal and noise covariances are unknown, rendering the classical Wiener filter inapplicable. The proposed method injects controlled Gaussian noise into training samples and optimizes the denoising matrix via least squares. Under a proportional asymptotic regime, the authors leverage the Convex Gaussian Min-Max Theorem (CGMT) to derive, for the first time, a closed-form expression for the generalization error, which is then used to optimize the noise injection strategy. Both theoretical analysis and experiments demonstrate that the resulting denoiser significantly outperforms the empirical Wiener filter across various settings and asymptotically approaches the optimal Wiener performance as the sample-to-dimension ratio increases.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Matrix & Tensor Methods
• Computer Vision: Learning & Optimization for CV

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

In the present paper we study the performance of linear denoisers for noisy data of the form $\mathbf{x} + \mathbf{z}$, where $\mathbf{x} \in \mathbb{R}^d$ is the desired data with zero mean and unknown covariance $\mathbfΣ$, and $\mathbf{z} \sim \mathcal{N}(0, \mathbfΣ_{\mathbf{z}})$ is additive noise. Since the covariance $\mathbfΣ$ is not known, the standard Wiener filter cannot be employed for denoising. Instead we assume we are given samples $\mathbf{x}_1,\dots,\mathbf{x}_n \in \mathbb{R}^d$ from the true distribution. A standard approach would then be to estimate $\mathbfΣ$ from the samples and use it to construct an ``empirical" Wiener filter. However, in this paper, motivated by the denoising step in diffusion models, we take a different approach whereby we train a linear denoiser $\mathbf{W}$ from the data itself. In particular, we synthetically construct noisy samples $\hat{\mathbf{x}}_i$ of the data by injecting the samples with Gaussian noise with covariance $\mathbfΣ_1 \neq \mathbfΣ_{\mathbf{z}}$ and find the best $\mathbf{W}$ that approximates $\mathbf{W}\hat{\mathbf{x}}_i \approx \mathbf{x}_i$ in a least-squares sense. In the proportional regime $\frac{n}{d} \rightarrow κ> 1$ we use the {\it Convex Gaussian Min-Max Theorem (CGMT)} to analytically find the closed form expression for the generalization error of the denoiser obtained from this process. Using this expression one can optimize over $\mathbfΣ_1$ to find the best possible denoiser. Our numerical simulations show that our denoiser outperforms the ``empirical" Wiener filter in many scenarios and approaches the optimal Wiener filter as $κ\rightarrow\infty$.

Problem

• linear denoising
• unknown covariance
• proportional regime
• generalization error
• additive Gaussian noise

Innovation

• linear denoiser
• proportional regime
• Convex Gaussian Min-Max Theorem
• empirical Wiener filter
• generalization error