More Haste, Less Speed: Weaker Single-Layer Watermark Improves Distortion-Free Watermark Ensembles

2026-02-12

AI Summary

This work addresses the limitation of existing watermarking methods, where excessively strong single-layer watermarks reduce token distribution entropy and thereby degrade the effectiveness of multi-layer ensembles. To overcome this, we propose a novel paradigm of “weak per-layer, strong overall” watermarking: by attenuating individual layer watermarks, we preserve both distributional entropy and the proportion of green-list tokens, enhancing overall detectability and robustness. Grounded in information-theoretic analysis, we derive theoretical bounds on watermark detectability, construct an integrated framework for weak single-layer watermarks, and elucidate the counterintuitive mechanism by which overly strong watermarks impair ensemble performance. Experimental results demonstrate that our approach effectively mitigates signal attenuation and significantly outperforms existing strong-watermark baselines in both detectability and robustness.

Technology Category

• Machine Learning: Ensemble Methods
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Watermarking has emerged as a crucial technique for detecting and attributing content generated by large language models. While recent advancements have utilized watermark ensembles to enhance robustness, prevailing methods typically prioritize maximizing the strength of the watermark at every individual layer. In this work, we identify a critical limitation in this"stronger-is-better"approach: strong watermarks significantly reduce the entropy of the token distribution, which paradoxically weakens the effectiveness of watermarking in subsequent layers. We theoretically and empirically show that detectability is bounded by entropy and that watermark ensembles induce a monotonic decrease in both entropy and the expected green-list ratio across layers. To address this inherent trade-off, we propose a general framework that utilizes weaker single-layer watermarks to preserve the entropy required for effective multi-layer ensembling. Empirical evaluations demonstrate that this counter-intuitive strategy mitigates signal decay and consistently outperforms strong baselines in both detectability and robustness.

Problem

• watermarking
• entropy
• large language models
• watermark ensembles
• token distribution

Innovation

• watermark ensembles
• entropy preservation
• weak watermarking
• green-list ratio
• LLM watermarking