Model-Based Clustering with Sequential Outlier Identification using the Distribution of Mahalanobis Distances

2025-05-16

AI Summary

Outliers severely impair clustering structure identification, yet existing methods often rely on prespecified outlier proportions or strong distributional assumptions. This paper proposes outlierMBC—a model-based, iterative framework that jointly performs clustering and outlier detection without requiring prior knowledge of outlier prevalence. Its core innovation lies in fitting a Gaussian mixture model (GMM), computing scaled squared Mahalanobis distances, and empirically estimating their distribution; it then automatically determines the optimal number of outliers to remove by minimizing the discrepancy between this empirical distribution and a Beta reference distribution. outlierMBC integrates model-driven clustering, density-ordered sequential outlier removal, and statistical distributional validation. Extensive experiments on synthetic and real-world datasets demonstrate that outlierMBC significantly improves clustering accuracy and adaptive outlier identification, consistently outperforming state-of-the-art baseline methods.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Clustering
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

The presence of outliers can prevent clustering algorithms from accurately determining an appropriate group structure within a data set. We present outlierMBC, a model-based approach for sequentially removing outliers and clustering the remaining observations. Our method identifies outliers one at a time while fitting a multivariate Gaussian mixture model to data. Since it can be difficult to classify observations as outliers without knowing what the correct cluster structure is a priori, and the presence of outliers interferes with the process of modelling clusters correctly, we use an iterative method to identify outliers one by one. At each iteration, outlierMBC removes the observation with the lowest density and fits a Gaussian mixture model to the remaining data. The method continues to remove potential outliers until a pre-set maximum number of outliers is reached, then retrospectively identifies the optimal number of outliers. To decide how many outliers to remove, it uses the fact that the squared sample Mahalanobis distances of Gaussian distributed observations are Beta distributed when scaled appropriately. outlierMBC chooses the number of outliers which minimises a dissimilarity between this theoretical Beta distribution and the observed distribution of the scaled squared sample Mahalanobis distances. This means that our method both clusters the data using a Gaussian mixture model and implements a model-based procedure to identify the optimal outliers to remove without requiring the number of outliers to be pre-specified. Unlike leading methods in the literature, outlierMBC does not assume that the outliers follow a known distribution or that the number of outliers can be pre-specified. Moreover, outlierMBC performs strongly compared to these algorithms when applied to a range of simulated and real data sets.

Problem

• Identifies outliers in clustering without pre-specifying their number
• Uses iterative Mahalanobis distance analysis for outlier removal
• Combines Gaussian mixture modeling with sequential outlier detection

Innovation

• Sequential outlier removal using Mahalanobis distances
• Iterative Gaussian mixture model fitting
• Beta distribution comparison for optimal outlier selection