Consistent Kernel Change-Point Detection under m-Dependence for Text Segmentation

2025-10-03

AI Summary

Existing kernel-based change-point detection (KCPD) methods rely on the independence assumption, rendering them ill-suited for strongly dependent sequences such as natural language text. Method: This work establishes, for the first time, a consistency theory for KCPD under the $m$-dependence assumption—relaxing the classical independence requirement—and proposes a novel framework that integrates modern language model embeddings with $m$-dependent modeling to achieve weakly consistent localization of structural changes. Contribution/Results: Our method achieves significant improvements over state-of-the-art approaches across multiple text segmentation benchmarks. It further demonstrates robustness and practical utility in real-world scenarios, including segmentation of Taylor Swift’s tweets. This work provides the first theoretical guarantee for kernel methods applied to dependent sequences and advances the trustworthy deployment of KCPD in NLP downstream tasks—particularly paragraph segmentation.

Technology Category

• Machine Learning: Kernel Methods
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Kernel change-point detection (KCPD) has become a widely used tool for identifying structural changes in complex data. While existing theory establishes consistency under independence assumptions, real-world sequential data such as text exhibits strong dependencies. We establish new guarantees for KCPD under $m$-dependent data: specifically, we prove consistency in the number of detected change points and weak consistency in their locations under mild additional assumptions. We perform an LLM-based simulation that generates synthetic $m$-dependent text to validate the asymptotics. To complement these results, we present the first comprehensive empirical study of KCPD for text segmentation with modern embeddings. Across diverse text datasets, KCPD with text embeddings outperforms baselines in standard text segmentation metrics. We demonstrate through a case study on Taylor Swift's tweets that KCPD not only provides strong theoretical and simulated reliability but also practical effectiveness for text segmentation tasks.

Problem

• Extending kernel change-point detection to handle dependent sequential data
• Proving consistency guarantees for change-point detection under m-dependence
• Validating text segmentation performance using embeddings and simulations

Innovation

• Establishes consistency guarantees for m-dependent data
• Uses LLM-based simulation to validate theoretical asymptotics
• Demonstrates text segmentation effectiveness with modern embeddings