Learning Distances from Data with Normalizing Flows and Score Matching

2024-07-12

arXiv.org

AI Summary

DBD methods face three key bottlenecks in high-dimensional metric learning: (1) poor convergence of Fermat distances due to biased density estimation, (2) limited scalability of graph-based approaches, and (3) non-smooth geodesic paths. To address these, we propose the first differentiable joint density–path modeling framework. Our method innovatively integrates neural spline flows with denoising score matching for accurate, scalable density estimation; introduces variational path relaxation coupled with Riemannian geometric constraints to optimize smooth geodesics; and designs a dimension-adaptive Fermat distance to mitigate the curse of dimensionality. Experiments demonstrate significant improvements in DBD estimation accuracy and geodesic smoothness in high-dimensional spaces. The framework consistently enhances clustering, visualization, and generation-guided learning performance on both image and synthetic benchmarks.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements

Abstract

Density-based distances (DBDs) provide a principled approach to metric learning by defining distances in terms of the underlying data distribution. By employing a Riemannian metric that increases in regions of low probability density, shortest paths naturally follow the data manifold. Fermat distances, a specific type of DBD, have attractive properties, but existing estimators based on nearest neighbor graphs suffer from poor convergence due to inaccurate density estimates. Moreover, graph-based methods scale poorly to high dimensions, as the proposed geodesics are often insufficiently smooth. We address these challenges in two key ways. First, we learn densities using normalizing flows. Second, we refine geodesics through relaxation, guided by a learned score model. Additionally, we introduce a dimension-adapted Fermat distance that scales intuitively to high dimensions and improves numerical stability. Our work paves the way for the practical use of density-based distances, especially in high-dimensional spaces.

Problem

• Improving density-based distance estimation with normalizing flows
• Enhancing geodesic smoothness via score-guided relaxation
• Scaling Fermat distances to high dimensions adaptively

Innovation

• Learn densities using normalizing flows
• Refine geodesics via learned score model
• Introduce dimension-adapted Fermat distance