Manifold-Matching Autoencoders

2026-03-17

AI Summary

This work addresses the challenge of preserving intrinsic data manifold structure during dimensionality reduction with autoencoders by proposing an unsupervised regularization method. The approach guides the model to learn distance-preserving low-dimensional representations by minimizing the mean squared error between pairwise distances in the latent space and those in the input space. Unlike methods requiring coordinate alignment, this technique directly constrains distance relationships, offering both flexibility and scalability, and can be viewed as an efficient approximation to classical multidimensional scaling (MDS). Experimental results demonstrate that the proposed method significantly outperforms existing approaches in preserving nearest-neighbor relationships and topological structure, the latter evaluated via persistent homology.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Representation Learning for Vision
• Data Mining & Knowledge Management: Data Compression

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

We study a simple unsupervised regularization scheme for autoencoders called Manifold-Matching (MMAE): we align the pairwise distances in the latent space to those of the input data space by minimizing mean squared error. Because alignment occurs on pairwise distances rather than coordinates, it can also be extended to a lower-dimensional representation of the data, adding flexibility to the method. We find that this regularization outperforms similar methods on metrics based on preservation of nearest-neighbor distances and persistent homology-based measures. We also observe that MMAE provides a scalable approximation of Multi-Dimensional Scaling (MDS).

Problem

• Manifold-Matching
• autoencoders
• distance preservation
• dimensionality reduction
• unsupervised regularization

Innovation

• Manifold-Matching
• autoencoder
• distance preservation
• unsupervised regularization
• Multi-Dimensional Scaling