Watermark Forensics for Generative Models: An Information-Theoretic Perspective

2026-07-14

AI Summary

This work addresses watermark forensics in generative model outputs beyond mere detection, enabling user attribution, hidden message extraction, and edit localization. Grounded in information theory, it introduces an information profile ν(t) to quantify the leakage of secret information by each token and formulates a hierarchy of information requirements for forensic tasks. The study establishes, for the first time, a tight entropy-rate law for multi-user attribution, revealing fundamental gaps among detection, attribution, and extraction. It further proposes an optimal decoder based on a surprisal-thresholding mechanism. Theoretically, attributing to N users requires Θ(log N/h) tokens, while extracting ℓ bits demands Θ(ℓ/h). Experiments on GPT-2, Pythia-410M, and Qwen2.5 confirm the accuracy of these theoretical predictions.

Abstract

A watermark in a generative model's output is usually asked only whether a text is machine-made. The same mark can do more: attribute it to the user who produced it, extract a hidden payload, or localize the part that survives editing. These form a forensic ladder, and we ask what each rung costs in the sample length $n$. One object organizes the answers. Let $S$ be the secret the mark carries (a user's identity or payload), and let the information profile $ν(t)=I(S;X_t\mid X_{<t})$ record how much the $t$-th token reveals about $S$ given the earlier ones. Its total mass pays for attribution and extraction; how that mass is spread pays for localization; and detection alone is paid for not by information but by presence, the distance from the marked to the unmarked distribution. The literature's two quality models, a mark subtle on every token and one that stamps a few tokens loudly, are two incomparable ways of capping this profile. Our main theorem settles the ladder's entropy column. For statistically distortion-free schemes, attributing a text to one of $N$ users costs $Θ(\log N/h)$ tokens over every stationary-ergodic source of entropy rate $h$, sharp to a $(1+o(1))$ factor: to our knowledge the first tight entropy-rate law for multi-user attribution (via exact alignment). The natural collision-counting analysis overcharges without bound; only a decoder thresholding each candidate by its own realized surprisal attains the rate while almost never implicating an innocent user. A matching converse makes the law two-sided, and extraction of an $\ell$-bit payload costs $Θ(\ell/h)$. Two gaps are real, not modeling artifacts: a $Θ(\log N)$-token window in which a text is provably machine-made yet unattributable, and a footprint-resolution uncertainty principle. Experiments on GPT-2, Pythia-410M, and Qwen2.5 recover the predicted constants.

Problem

• watermark forensics
• generative models
• information-theoretic
• attribution
• payload extraction

Innovation

• watermark forensics
• information-theoretic analysis
• generative model attribution
• entropy-rate bounds
• realized surprisal decoding