Spectral Clustering in Birthday Paradox Time

2026-01-09

arXiv.org

AI Summary

This work addresses the problem of efficiently determining the cluster membership of vertices in $(k, \phi, \varepsilon)$-clusterable graphs. The authors propose a hybrid representation based on logarithmic-length random walks, combined with nearest neighbor search to enable fast cluster membership queries. By constructing a representation that achieves the optimal sampling complexity dictated by the birthday paradox lower bound, their method is the first to theoretically guarantee that query complexity decreases as the number of clusters $k$ increases, thereby resolving a longstanding inconsistency between conceptual clarity and computational efficiency in prior approaches. The resulting clustering oracle achieves query time $\approx (n/k)^{1/2 + O(\varepsilon/\phi^2)}$ and space complexity $k \cdot (n/k)^{1/2 + O(\varepsilon/\phi^2)}$, matching the known theoretical lower bounds.

Technology Category

• Machine Learning: Clustering
• Search and Optimization: Mixed Discrete/Continuous Search
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Given a vertex in a $(k, \varphi, \epsilon)$-clusterable graph, i.e. a graph whose vertex set can be partitioned into a disjoint union of $\varphi$-expanders of size $\approx n/k$ with outer conductance bounded by $\epsilon$, can one quickly tell which cluster it belongs to? This question goes back to the expansion testing problem of Goldreich and Ron'11. For $k=2$ a sample of $\approx n^{1/2+O(\epsilon/\varphi^2)}$ logarithmic length walks from a given vertex approximately determines its cluster membership by the birthday paradox: two vertices whose random walk samples are `close'are likely in the same cluster. The study of the general case $k>2$ was initiated by Czumaj, Peng and Sohler [STOC'15], and the works of Chiplunkar et al. [FOCS'18], Gluch et al. [SODA'21] showed that $\approx \text{poly}(k)\cdot n^{1/2+O(\epsilon/\varphi^2)}$ random walk samples suffice for general $k$. This matches the $k=2$ result up to polynomial factors in $k$, but creates a conceptual inconsistency: if the birthday paradox is the guiding phenomenon, then the query complexity should decrease with the number of clusters $k$! Since clusters have size $\approx n/k$, we expect to need $\approx (n/k)^{1/2+O(\epsilon/\varphi^2)}$ random walk samples, which decreases with $k$. We design a novel representation of vertices in a $(k, \varphi, \epsilon)$-clusterable graph by a mixture of logarithmic length walks. This representation uses the optimal $\approx (n/k)^{1/2+O(\epsilon/\varphi^2)}$ walks per vertex, and allows for a fast nearest neighbor search: given $k$ vertices representing the clusters, we can find the cluster of a given query vertex $x$ using nearly linear time in the representation size of $x$. This gives a clustering oracle with query time $\approx (n/k)^{1/2+O(\epsilon/\varphi^2)}$ and space complexity $k\cdot (n/k)^{1/2+O(\epsilon/\varphi^2)}$, matching the birthday paradox bound.

Problem

• Spectral Clustering
• Birthday Paradox
• Clusterable Graphs
• Random Walks
• Query Complexity

Innovation

• spectral clustering
• birthday paradox
• random walks
• clustering oracle
• expanders