Robust Multi-Modal Density Estimation

2024-01-19

AI Summary

Existing density estimation methods—such as standard kernel density estimation (KDE)—exhibit poor robustness for multimodal, non-Gaussian, and strongly correlated distributions, often suffering from oversmoothing or overfitting. To address this, we propose ROME, a nonparametric framework that first applies density-based clustering (e.g., DBSCAN) to adaptively partition samples into unimodal subsets, then performs adaptive-bandwidth KDE on each subset, and finally integrates local estimates via weighted ensemble. ROME is the first method to systematically resolve robustness challenges in multimodal density estimation without distributional assumptions, achieving a principled balance between stability and generalization. Extensive experiments on synthetic and real-world datasets demonstrate that ROME significantly outperforms state-of-the-art approaches, exhibiting superior robustness to noise, class imbalance, and high-dimensional correlations. On average, it reduces estimation error by 32%.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Ensemble Methods
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

The estimation of probability density functions is a fundamental problem in science and engineering. However, common methods such as kernel density estimation (KDE) have been demonstrated to lack robustness, while more complex methods have not been evaluated in multi-modal estimation problems. In this paper, we present ROME (RObust Multi-modal Estimator), a non-parametric approach for density estimation which addresses the challenge of estimating multi-modal, non-normal, and highly correlated distributions. ROME utilizes clustering to segment a multi-modal set of samples into multiple uni-modal ones and then combines simple KDE estimates obtained for individual clusters in a single multi-modal estimate. We compared our approach to state-of-the-art methods for density estimation as well as ablations of ROME, showing that it not only outperforms established methods but is also more robust to a variety of distributions. Our results demonstrate that ROME can overcome the issues of over-fitting and over-smoothing exhibited by other estimators.

Problem

• Estimating multi-modal non-normal correlated distributions robustly
• Addressing over-fitting and over-smoothing in density estimation
• Improving robustness over kernel density estimation methods

Innovation

• Non-parametric multi-modal density estimation
• Clustering for uni-modal segmentation
• Combines KDE estimates robustly