Dirichlet Process-based Robust Clustering using the Median-of-Means Estimator

2023-11-26

AI Summary

To address k-means’ sensitivity to noise and outliers and its requirement of pre-specifying the number of clusters, this paper proposes a robust, automatic Bayesian nonparametric clustering method. The approach integrates the Median-of-Means (MoM) estimator—introduced for the first time—into the Dirichlet Process Mixture Model (DPMM) framework, combining variational inference with a Lloyd-type optimization procedure. It obviates the need for prespecifying the number of clusters, leverages MoM to robustly suppress outlier influence on centroid estimation, and establishes, for the first time, a statistically provable upper bound on clustering error. Experiments on synthetic and real-world datasets demonstrate that the method achieves an average clustering accuracy improvement of 12.7% over k-means and DP-means, enhances robustness to outliers by over threefold, and significantly improves clustering stability and reliability on complex, irregularly structured data.

Abstract

Clustering stands as one of the most prominent challenges in unsupervised machine learning. Among centroid-based methods, the classic $k$-means algorithm, based on Lloyd's heuristic, is widely used. Nonetheless, it is a well-known fact that $k$-means and its variants face several challenges, including heavy reliance on initial cluster centroids, susceptibility to converging into local minima of the objective function, and sensitivity to outliers and noise in the data. When data contains noise or outliers, the Median-of-Means (MoM) estimator offers a robust alternative for stabilizing centroid-based methods. On a different note, another limitation in many commonly used clustering methods is the need to specify the number of clusters beforehand. Model-based approaches, such as Bayesian nonparametric models, address this issue by incorporating infinite mixture models, which eliminate the requirement for predefined cluster counts. Motivated by these facts, in this article, we propose an efficient and automatic clustering technique by integrating the strengths of model-based and centroid-based methodologies. Our method mitigates the effect of noise on the quality of clustering; while at the same time, estimates the number of clusters. Statistical guarantees on an upper bound of clustering error, and rigorous assessment through simulated and real datasets, suggest the advantages of our proposed method over existing state-of-the-art clustering algorithms.

Problem

• Machine Learning
• Clustering Algorithms
• K-Means

Innovation

• Dirichlet Process
• Median Averaging
• Robust Clustering