Joint Representation Learning and Clustering via Gradient-Based Manifold Optimization

2026-04-15

AI Summary

This work addresses the challenge of clustering high-dimensional data, which is often hindered by the curse of dimensionality. The authors propose a unified framework based on gradient-based manifold optimization that jointly learns a dimensionality reduction mapping—such as a linear projection or a neural network—and a clustering structure, exemplified by Gaussian mixture models. To the best of our knowledge, this is the first approach to integrate gradient manifold optimization into an end-to-end joint learning scheme for representation and clustering, effectively achieving unsupervised linear discriminant analysis–like performance. Extensive experiments on both synthetic data and the MNIST image dataset demonstrate that the proposed method significantly outperforms state-of-the-art clustering algorithms, confirming its efficacy and superiority.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Representation Learning for Vision
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Clustering and dimensionality reduction have been crucial topics in machine learning and computer vision. Clustering high-dimensional data has been challenging for a long time due to the curse of dimensionality. For that reason, a more promising direction is the joint learning of dimension reduction and clustering. In this work, we propose a Manifold Learning Framework that learns dimensionality reduction and clustering simultaneously. The proposed framework is able to jointly learn the parameters of a dimension reduction technique (e.g. linear projection or a neural network) and cluster the data based on the resulting features (e.g. under a Gaussian Mixture Model framework). The framework searches for the dimension reduction parameters and the optimal clusters by traversing a manifold,using Gradient Manifold Optimization. The obtained The proposed framework is exemplified with a Gaussian Mixture Model as one simple but efficient example, in a process that is somehow similar to unsupervised Linear Discriminant Analysis (LDA). We apply the proposed method to the unsupervised training of simulated data as well as a benchmark image dataset (i.e. MNIST). The experimental results indicate that our algorithm has better performance than popular clustering algorithms from the literature.

Problem

• clustering
• dimensionality reduction
• high-dimensional data
• joint learning
• curse of dimensionality

Innovation

• Manifold Optimization
• Joint Representation Learning
• Clustering
• Dimensionality Reduction
• Gaussian Mixture Model