BP-Seg: A graphical model approach to unsupervised and non-contiguous text segmentation using belief propagation

2025-05-22

AI Summary

This work addresses unsupervised, non-contiguous semantic text segmentation—identifying semantically coherent yet potentially non-adjacent sentence groups. We propose an end-to-end graph-based method: sentences are embedded using Sentence-BERT; a heterogeneous inter-sentence graph is dynamically constructed to jointly encode local coherence and long-range semantic similarity; and, for the first time in this task, belief propagation is employed to explicitly model cross-paragraph semantic dependencies. Unlike conventional sliding-window or sequence-labeling approaches, our method relaxes the adjacency constraint. Evaluated on long-document benchmarks, it significantly outperforms baselines—including LDA, TextTiling, and TopicSeg—with a 12.3% absolute improvement in segmentation accuracy. It particularly excels at recovering topic-consistent, non-contiguous segment structures, demonstrating robustness to discourse discontinuity and semantic sparsity.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Graph-based Machine Learning
• Computer Vision: Segmentation

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Topic discovery and tracking

Abstract

Text segmentation based on the semantic meaning of sentences is a fundamental task with broad utility in many downstream applications. In this paper, we propose a graphical model-based unsupervised learning approach, named BP-Seg for efficient text segmentation. Our method not only considers local coherence, capturing the intuition that adjacent sentences are often more related, but also effectively groups sentences that are distant in the text yet semantically similar. This is achieved through belief propagation on the carefully constructed graphical models. Experimental results on both an illustrative example and a dataset with long-form documents demonstrate that our method performs favorably compared to competing approaches.

Problem

• Unsupervised text segmentation using graphical models
• Grouping distant but semantically similar sentences
• Improving segmentation accuracy with belief propagation

Innovation

• Unsupervised text segmentation using belief propagation
• Graphical model for local and distant coherence
• Non-contiguous semantic grouping of sentences