WaterSeeker: Pioneering Efficient Detection of Watermarked Segments in Large Documents

2024-09-08

AI Summary

To address the low efficiency and coarse-grained localization (currently limited to document-level only) of watermark detection caused by localized embedding of large language model (LLM)-generated text in long documents, this paper proposes a two-stage watermark detection paradigm. First, a lightweight statistical anomaly detector pre-screens suspicious regions; then, a sliding-window-based local traversal combined with an improved confidence verification mechanism enables precise paragraph-level localization. To our knowledge, this is the first method capable of localizing fine-grained watermark segments—down to as few as 16 tokens—thereby significantly enhancing the interpretability of AI-generated content detection. Experiments demonstrate that the method maintains over 95% detection accuracy on ten-thousand-word documents while achieving an 8.2× speedup over baseline approaches, enabling real-time analysis.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Watermarking algorithms for large language models (LLMs) have attained high accuracy in detecting LLM-generated text. However, existing methods primarily focus on distinguishing fully watermarked text from non-watermarked text, overlooking real-world scenarios where LLMs generate only small sections within large documents. In this scenario, balancing time complexity and detection performance poses significant challenges. This paper presents WaterSeeker, a novel approach to efficiently detect and locate watermarked segments amid extensive natural text. It first applies an efficient anomaly extraction method to preliminarily locate suspicious watermarked regions. Following this, it conducts a local traversal and performs full-text detection for more precise verification. Theoretical analysis and experimental results demonstrate that WaterSeeker achieves a superior balance between detection accuracy and computational efficiency. Moreover, its localization capability lays the foundation for building interpretable AI detection systems. Our code is available at https://github.com/THU-BPM/WaterSeeker.

Problem

• Watermarking
• Large Language Models
• Text Attribution

Innovation

• WaterSeeker
• AI-generated content detection
• Watermark identification