Semidefinite programming relaxations and debiasing for MAXCUT-based clustering

2024-01-16

AI Summary

This paper studies clustering in sparse, high-dimensional sub-Gaussian mixture models under the small-sample regime, focusing on partial recovery in low signal-to-noise ratio (SNR) settings using only a few high-quality features. We propose BalancedSDP, a novel estimator that formulates clustering as a feature-weighted MAXCUT problem and solves it via semidefinite programming (SDP) relaxation. Our key contributions are threefold: (i) We establish, for the first time without assuming bounded trace difference of covariances, that the misclassification error decays exponentially in the square of the SNR ($s^2$); (ii) We develop a unified statistical analysis framework encompassing SDP1, BalancedSDP, and spectral clustering; (iii) We theoretically characterize sharp phase-transition thresholds for both balanced and imbalanced mixtures, and corroborate via simulations that partial recoverability is SNR-driven and aligns precisely with our theoretical predictions.

Technology Category

• Machine Learning: Clustering
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In this paper, we consider the problem of partitioning a small data sample of size $n$ drawn from a mixture of 2 sub-gaussian distributions in $mathbb{R}^p$. We consider semidefinite programming relaxations of an integer quadratic program that is formulated as finding the maximum cut on a graph, where edge weights in the cut represent dissimilarity scores between two nodes based on their $p$ features. We are interested in the case that individual features are of low average quality $gamma$, and we want to use as few of them as possible to correctly partition the sample. Denote by $Delta^2:=p gamma$ the $ell_2^2$ distance between two centers (mean vectors) in $mathbb{R}^p$. The goal is to allow a full range of tradeoffs between $n, p, gamma$ in the sense that partial recovery (success rate $<100%$) is feasible once the signal to noise ratio $s^2 := min{np gamma^2, Delta^2}$ is lower bounded by a constant. For both balanced and unbalanced cases, we allow each population to have distinct covariance structures with diagonal matrices as special cases. In the present work, (a) we provide a unified framework for analyzing three computationally efficient algorithms, namely, SDP1, BalancedSDP, and Spectral clustering; and (b) we prove that the misclassification error decays exponentially with respect to the SNR $s^2$ for SDP1. Moreover, for balanced partitions, we design an estimator {f BalancedSDP} with a superb debiasing property. Indeed, with this new estimator, we remove an assumption (A2) on bounding the trace difference between the two population covariance matrices while proving the exponential error bound as stated above. These estimators and their statistical analyses are novel to the best of our knowledge. We provide simulation evidence illuminating the theoretical predictions.

Problem

• Partitioning data from 2 sub-gaussian distributions using MAXCUT-based clustering.
• Analyzing SDP relaxations for clustering with low-quality features.
• Developing debiased estimators for balanced partitions with distinct covariance structures.

Innovation

• Semidefinite programming relaxations for MAXCUT-based clustering
• Debiasing estimator for balanced partitions
• Exponential error decay with respect to SNR