Deep Generative Clustering with VAEs and Expectation-Maximization

2025-01-13

AI Summary

To address the challenge of jointly optimizing clustering performance and generative capability in deep clustering, this paper proposes VAE-EM: a deep generative clustering framework that natively integrates a variational autoencoder (VAE) into the expectation-maximization (EM) paradigm. Unlike conventional approaches relying on Gaussian mixture model (GMM) priors or auxiliary regularization terms, VAE-EM directly models each cluster in the latent space as a learnable probability distribution. It jointly optimizes clustering structure learning and cluster-conditional sample generation via alternating maximization of the evidence lower bound (ELBO) and cluster posterior estimation. On MNIST and FashionMNIST, VAE-EM achieves significantly higher clustering accuracy than current state-of-the-art methods. Moreover, it enables semantic-consistent, cluster-specific generation of novel samples—marking the first end-to-end framework that unifies hard clustering objectives with explicit generative modeling within a single coherent optimization procedure.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

We propose a novel deep clustering method that integrates Variational Autoencoders (VAEs) into the Expectation-Maximization (EM) framework. Our approach models the probability distribution of each cluster with a VAE and alternates between updating model parameters by maximizing the Evidence Lower Bound (ELBO) of the log-likelihood and refining cluster assignments based on the learned distributions. This enables effective clustering and generation of new samples from each cluster. Unlike existing VAE-based methods, our approach eliminates the need for a Gaussian Mixture Model (GMM) prior or additional regularization techniques. Experiments on MNIST and FashionMNIST demonstrate superior clustering performance compared to state-of-the-art methods.

Problem

• Deep Learning Clustering
• Data Grouping
• Feature Understanding

Innovation

• Variational Autoencoders
• Expectation-Maximization Framework
• Clustering Performance