Signature filtering: a lightweight enhancement for statistical watermark detection in large language models

2026-06-16

AI Summary

Existing watermark detection methods for large language models suffer significant performance degradation under weak watermark signals, text duplication, or post-generation editing. This work proposes a signature filtering mechanism that automatically identifies and removes “signature” tokens—those undermining watermark reliability—during detection, without altering the watermark embedding or text generation pipeline. It is the first approach to learn such signature tokens via mixed-integer linear programming, integrating n-gram modeling with statistical hypothesis testing. The framework supports diverse watermarking schemes and language models, offering theoretical robustness guarantees against multiple attack types. Experiments demonstrate that, in weak-signal settings, the method boosts detection rates from 8–31% to 78–99% while maintaining an extremely low false positive rate, and consistently outperforms state-of-the-art detectors under strong perturbations.

Abstract

Statistical watermarks help organizations attribute large language model (LLM) outputs, yet existing detectors often struggle when watermark signals are weak, texts are repetitive, or watermarks are edited. We propose signature filtering, a detection-time module that enhances watermark detection without modifying watermark embedding and text generation. It learns a small set of ``signature'' tokens whose presence makes watermark tests unreliable, and removes these tokens before detection. The signatures are obtained by solving a mixed-integer linear program on a small training set, with constraints that maximize the true positive rate. We additionally derive finite-sample and asymptotic bounds under several attacker models (color-blind, color-adaptive, and distributionally correlated). On four well-known watermark families (Kgw, Sweet, Unigram, Exp), four benchmark corpora (C4, MBPP, HumanEval, Code-Search-Net), and six LLMs (Opt-1.3b, Opt-6.7b, Llama2-13b, Llama3.1-8b, Qwen2.5-14b, Phi-3-medium-14b), 2- and 3-gram signatures raise detection rates in weak-signal and low-entropy settings from 8~31% without filtering to 78~99% with filtering, while keeping false positives controllable and often negligible. In stress tests where we scramble sentences and perturb 25~50% of tokens by dilution, deletions, and substitutions, 2-gram filters for Kgw-style watermarks preserve most of the clean-text detection gains, often matching or outperforming the advanced WinMax watermark detector. Signature filtering thus provides a simple, scalable, and model-agnostic add-on to strengthen watermark-based provenance checks for LLM text in information processing workflows.

Problem

• statistical watermark detection
• large language models
• weak watermark signals
• text repetition
• watermark editing

Innovation

• signature filtering
• statistical watermarking
• watermark detection
• large language models
• mixed-integer linear programming