Learning a distance measure from the information-estimation geometry of data

2025-10-02

AI Summary

This work addresses the challenge that conventional signal similarity metrics fail to adapt to the intrinsic probabilistic geometry of data. We propose the Information–Estimation Metric (IEM), a novel distance function grounded in differences between denoising error vectors. IEM establishes, for the first time, a theoretical connection between the denoising error field and the score vector field, enabling explicit computation via a one-dimensional integral—thereby ensuring both theoretical soundness and practical implementability. Under Gaussian assumptions, IEM reduces to the Mahalanobis distance; more generally, it adapts to both local and global geometric structures of complex distributions. Crucially, IEM can be directly approximated using pre-trained denoising models—e.g., diffusion models—without additional training. When learned on ImageNet, IEM achieves or surpasses state-of-the-art supervised image quality assessment methods on human perceptual quality prediction tasks, demonstrating its effectiveness and strong generalization capability.

Technology Category

• Machine Learning: Information Theory
• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We introduce the Information-Estimation Metric (IEM), a novel form of distance function derived from an underlying continuous probability density over a domain of signals. The IEM is rooted in a fundamental relationship between information theory and estimation theory, which links the log-probability of a signal with the errors of an optimal denoiser, applied to noisy observations of the signal. In particular, the IEM between a pair of signals is obtained by comparing their denoising error vectors over a range of noise amplitudes. Geometrically, this amounts to comparing the score vector fields of the blurred density around the signals over a range of blur levels. We prove that the IEM is a valid global metric and derive a closed-form expression for its local second-order approximation, which yields a Riemannian metric. For Gaussian-distributed signals, the IEM coincides with the Mahalanobis distance. But for more complex distributions, it adapts, both locally and globally, to the geometry of the distribution. In practice, the IEM can be computed using a learned denoiser (analogous to generative diffusion models) and solving a one-dimensional integral. To demonstrate the value of our framework, we learn an IEM on the ImageNet database. Experiments show that this IEM is competitive with or outperforms state-of-the-art supervised image quality metrics in predicting human perceptual judgments.

Problem

• Deriving a distance metric from information-estimation geometry of data distributions
• Computing distances by comparing denoising error vectors across noise levels
• Learning adaptive metrics that predict human perceptual judgments competitively

Innovation

• Distance metric derived from denoising error vectors
• Closed-form Riemannian metric from local approximation
• Learned denoiser computes metric via one-dimensional integral