QuantileMark: A Message-Symmetric Multi-bit Watermark for LLMs

2026-04-15

AI Summary

This work addresses the issue that existing multi-bit watermarking methods for large language models compromise message symmetry under low-entropy decoding, causing generation quality and verification performance to depend on the embedded content. To resolve this, the authors propose QuantileMark, a white-box multi-bit watermarking approach that partitions the cumulative probability interval [0,1) into M equal-mass bins and consistently samples tokens from the bin corresponding to the target message. This strategy formally achieves message unbiasedness for the first time, preserving the original generation distribution in expectation. Leveraging equal-mass binning, teacher-forced verifier reconstruction, and posterior aggregation detection, QuantileMark significantly outperforms strong baselines on C4 continuation and LFQA tasks, achieving higher bit recovery rates and detection robustness while exerting negligible impact on text quality.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

As large language models become standard backends for content generation, practical provenance increasingly requires multi-bit watermarking. In provider-internal deployments, a key requirement is message symmetry: the message itself should not systematically affect either text quality or verification outcomes. Vocabulary-partition watermarks can break message symmetry in low-entropy decoding: some messages are assigned most of the probability mass, while others are forced to use tail tokens. This makes embedding quality and message decoding accuracy message-dependent. We propose QuantileMark, a white-box multi-bit watermark that embeds messages within the continuous cumulative probability interval $[0, 1)$. At each step, QuantileMark partitions this interval into $M$ equal-mass bins and samples strictly from the bin assigned to the target symbol, ensuring a fixed $1/M$ probability budget regardless of context entropy. For detection, the verifier reconstructs the same partition under teacher forcing, computes posteriors over latent bins, and aggregates evidence for verification. We prove message-unbiasedness, a property ensuring that the base distribution is recovered when averaging over messages. This provides a theoretical foundation for generation-side symmetry, while the equal-mass design additionally promotes uniform evidence strength across messages on the detection side. Empirical results on C4 continuation and LFQA show improved multi-bit recovery and detection robustness over strong baselines, with negligible impact on generation quality. Our code is available at GitHub (https://github.com/zzzjunlin/QuantileMark).

Problem

• message symmetry
• multi-bit watermarking
• large language models
• watermarking
• generation quality

Innovation

• message symmetry
• multi-bit watermarking
• equal-mass quantile partitioning
• message-unbiasedness
• LLM provenance