Scalable Uncertainty Quantification for Black-Box Density-Based Clustering

2026-03-03

AI Summary

This work addresses the lack of scalable and reliable uncertainty quantification in black-box density-based clustering by introducing, for the first time, the martingale posterior paradigm to this setting. The proposed framework enables end-to-end propagation of uncertainty—from density estimation through to cluster structure—while providing high-frequency consistency guarantees. By integrating neural density estimators with GPU-accelerated parallel computation, the method efficiently handles high-dimensional and irregularly structured data. Experimental evaluations on both synthetic and real-world datasets demonstrate that the approach achieves strong scalability, solid theoretical guarantees, and practical effectiveness, offering a principled and computationally feasible solution for uncertainty-aware density clustering.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

We introduce a novel framework for uncertainty quantification in clustering. By combining the martingale posterior paradigm with density-based clustering, uncertainty in the estimated density is naturally propagated to the clustering structure. The approach scales effectively to high-dimensional and irregularly shaped data by leveraging modern neural density estimators and GPU-friendly parallel computation. We establish frequentist consistency guarantees and validate the methodology on synthetic and real data.

Problem

• uncertainty quantification
• density-based clustering
• scalability
• black-box models
• clustering uncertainty

Innovation

• uncertainty quantification
• density-based clustering
• martingale posterior
• neural density estimation
• scalable clustering