Convergence of Mean Shift Algorithms for Large Bandwidths and Simultaneous Accurate Clustering

2025-06-24

AI Summary

Mean-shift convergence lacks rigorous theoretical justification under general conditions—particularly in high-dimensional spaces and with large bandwidths. This paper establishes, for the first time, a global convergence guarantee for mean-shift with arbitrary radially symmetric, strictly positive definite kernels (including the Gaussian kernel) in the large-bandwidth regime. Our analysis unifies Schoenberg’s and Bernstein’s kernel characterizations, kernel density estimation, Lipschitz gradient bounds, the Łojasiewicz inequality, spectral properties of positive definite kernels, and convex-hull invariance of iterative trajectories into a single analytical framework. Crucially, the proof imposes no dimensional constraints. This result substantially broadens the theoretical foundations of density-based clustering, while simultaneously enhancing the accuracy and stability of clustering outcomes.

Technology Category

• Machine Learning: Kernel Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

The mean shift (MS) is a non-parametric, density-based, iterative algorithm that has prominent usage in clustering and image segmentation. A rigorous proof for its convergence in full generality remains unknown. Two significant steps in this direction were taken in the paper cite{Gh1}, which proved that for extit{sufficiently large bandwidth}, the MS algorithm with the Gaussian kernel always converges in any dimension, and also by the same author in cite{Gh2}, proved that MS always converges in one dimension for kernels with differentiable, strictly decreasing, convex profiles. In the more recent paper cite{YT}, they have proved the convergence in more generality, extit{ without any restriction on the bandwidth}, with the assumption that the KDE $f$ has a continuous Lipschitz gradient on the closure of the convex hull of the trajectory of the iterated sequence of the mode estimate, and also satisfies the Łojasiewicz property there. The main theoretical result of this paper is a generalization of those of cite{Gh1}, where we show that (1) for extit{ sufficiently large bandwidth} convergence is guaranteed in any dimension with extit{any radially symmetric and strictly positive definite kernels}. The proof uses two alternate characterizations of radially symmetric positive definite smooth kernels by Schoenberg and Bernstein cite{Fass}, and borrows some steps from the proofs in cite{Gh1}. Although the authors acknowledge that the result in that paper is more restrictive than that of cite{YT} due to the lower bandwidth limit, it uses a different set of assumptions than cite{YT}, and the proof technique is different.

Problem

• Proving convergence of Mean Shift for large bandwidths
• Generalizing convergence to any radially symmetric kernels
• Extending convergence guarantees to any dimension

Innovation

• Convergence proven for large bandwidths
• Uses radially symmetric positive kernels
• Generalizes prior mean shift results