dgMARK: Decoding-Guided Watermarking for Diffusion Language Models

2026-01-30

AI Summary

This work addresses the lack of effective watermarking mechanisms for discrete diffusion language models (dLLMs) by proposing a plug-and-play text watermarking method that requires no modification to the model’s probability distribution. The approach leverages, for the first time, the sensitivity of dLLMs to demasking order as a watermark channel, embedding high-reward tokens that satisfy binary hash parity constraints through guided decoding sequences, augmented with a one-step lookahead strategy. During detection, a sliding-window statistical test enables robust watermark identification while remaining compatible with mainstream decoding strategies—including confidence-, entropy-, and margin-based ranking—and effectively resists common post-editing attacks such as insertion, deletion, substitution, and paraphrasing. Experimental results demonstrate that the method achieves high watermark robustness without compromising text generation quality, making it suitable for practical deployment.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Diffusion Models for Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

We propose dgMARK, a decoding-guided watermarking method for discrete diffusion language models (dLLMs). Unlike autoregressive models, dLLMs can generate tokens in arbitrary order. While an ideal conditional predictor would be invariant to this order, practical dLLMs exhibit strong sensitivity to the unmasking order, creating a new channel for watermarking. dgMARK steers the unmasking order toward positions whose high-reward candidate tokens satisfy a simple parity constraint induced by a binary hash, without explicitly reweighting the model's learned probabilities. The method is plug-and-play with common decoding strategies (e.g., confidence, entropy, and margin-based ordering) and can be strengthened with a one-step lookahead variant. Watermarks are detected via elevated parity-matching statistics, and a sliding-window detector ensures robustness under post-editing operations including insertion, deletion, substitution, and paraphrasing.

Problem

• watermarking
• diffusion language models
• unmasking order
• text generation
• content attribution

Innovation

• decoding-guided watermarking
• diffusion language models
• unmasking order
• parity constraint
• robust detection