Incremental Graph Construction Enables Robust Spectral Clustering of Texts

2026-03-03

AI Summary

This work addresses the limitations of standard k-nearest neighbor (k-NN) graphs in text spectral clustering, where low values of k often yield disconnected graphs, leading to degenerate clustering results and high sensitivity to hyperparameters. To overcome this, the authors propose an incremental k-NN graph construction method that ensures strict graph connectivity: as each new node is inserted, it is connected exclusively to its k nearest neighbors among previously inserted nodes. This approach naturally supports streaming document updates and integrates SentenceTransformer embeddings with Laplacian eigenmaps. Evaluated on six large-scale text clustering benchmarks, the method significantly outperforms conventional k-NN graphs at low k values while matching their performance at higher k, thereby enhancing both the robustness and practicality of spectral clustering for textual data.

Technology Category

• Machine Learning: Clustering
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Neighborhood graphs are a critical but often fragile step in spectral clustering of text embeddings. On realistic text datasets, standard $k$-NN graphs can contain many disconnected components at practical sparsity levels (small $k$), making spectral clustering degenerate and sensitive to hyperparameters. We introduce a simple incremental $k$-NN graph construction that preserves connectivity by design: each new node is linked to its $k$ nearest previously inserted nodes, which guarantees a connected graph for any $k$. We provide an inductive proof of connectedness and discuss implications for incremental updates when new documents arrive. We validate the approach on spectral clustering of SentenceTransformer embeddings using Laplacian eigenmaps across six clustering datasets from the Massive Text Embedding Benchmark.Compared to standard $k$-NN graphs, our method outperforms in the low-$k$ regime where disconnected components are prevalent, and matches standard $k$-NN at larger $k$.

Problem

• spectral clustering
• k-NN graph
• disconnected components
• text embeddings
• graph connectivity

Innovation

• incremental graph construction
• spectral clustering
• k-NN graph connectivity
• text embeddings
• Laplacian eigenmaps