Optimal Estimation of Watermark Proportions in Hybrid AI-Human Texts

2025-06-27

AI Summary

This work addresses the problem of optimally estimating the watermark proportion in hybrid-text corpora comprising both human-written and large language model (LLM)-generated content. Existing methods struggle to accurately identify and quantify the watermark fraction, constituting a fundamental bottleneck. We first establish that watermarking schemes based on continuous pivotal statistics possess proportion-parameter identifiability—a previously unproven property—and construct a minimax-optimal estimator achieving the theoretical lower bound. Our method integrates pivotal-statistic modeling, nonparametric estimation, and information-theoretic lower-bound analysis, and is compatible with diverse unbiased watermark detectors. Extensive evaluation on synthetic data and real-world hybrid texts generated by open-source LLMs demonstrates that the proposed estimator achieves high accuracy and strong robustness, with estimation error consistently approaching the fundamental performance limit.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: State Estimation
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Text watermarks in large language models (LLMs) are an increasingly important tool for detecting synthetic text and distinguishing human-written content from LLM-generated text. While most existing studies focus on determining whether entire texts are watermarked, many real-world scenarios involve mixed-source texts, which blend human-written and watermarked content. In this paper, we address the problem of optimally estimating the watermark proportion in mixed-source texts. We cast this problem as estimating the proportion parameter in a mixture model based on emph{pivotal statistics}. First, we show that this parameter is not even identifiable in certain watermarking schemes, let alone consistently estimable. In stark contrast, for watermarking methods that employ continuous pivotal statistics for detection, we demonstrate that the proportion parameter is identifiable under mild conditions. We propose efficient estimators for this class of methods, which include several popular unbiased watermarks as examples, and derive minimax lower bounds for any measurable estimator based on pivotal statistics, showing that our estimators achieve these lower bounds. Through evaluations on both synthetic data and mixed-source text generated by open-source models, we demonstrate that our proposed estimators consistently achieve high estimation accuracy.

Problem

• Estimating watermark proportion in mixed human-AI texts
• Identifying conditions for watermark proportion identifiability
• Developing efficient estimators for continuous pivotal watermarks

Innovation

• Estimating watermark proportion using pivotal statistics
• Identifiable proportion under mild conditions
• Efficient estimators achieving minimax lower bounds