WorldCup Sampling for Multi-bit LLM Watermarking

2026-02-02

AI Summary

This work addresses the limitations of existing multi-bit watermarking methods for large language models, which rely on seed-based guidance and consequently suffer from indirect information embedding, limited capacity, and suboptimal decoding performance. The authors model text generation as a communication channel and propose a hierarchical competitive sampling mechanism guided by complementary signals to directly embed information bits during token selection. To preserve textual quality, they integrate entropy-aware modulation, and further enhance robustness through a confidence-aware decoding strategy. The resulting framework significantly outperforms current approaches across multiple dimensions—watermark capacity, detectability, robustness, text quality, and decoding efficiency—enabling highly effective and reliable multi-bit watermark embedding and extraction.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

As large language models (LLMs) generate increasingly human-like text, watermarking offers a promising solution for reliable attribution beyond mere detection. While multi-bit watermarking enables richer provenance encoding, existing methods largely extend zero-bit schemes through seed-driven steering, leading to indirect information flow, limited effective capacity, and suboptimal decoding. In this paper, we propose WorldCup, a multi-bit watermarking framework for LLMs that treats sampling as a natural communication channel and embeds message bits directly into token selection via a hierarchical competition mechanism guided by complementary signals. Moreover, WorldCup further adopts entropy-aware modulation to preserve generation quality and supports robust message recovery through confidence-aware decoding. Comprehensive experiments show that WorldCup achieves a strong balance across capacity, detectability, robustness, text quality, and decoding efficiency, consistently outperforming prior baselines and laying a solid foundation for future LLM watermarking studies.

Problem

• LLM watermarking
• multi-bit watermarking
• token sampling
• information embedding
• generation quality

Innovation

• multi-bit watermarking
• hierarchical competition
• entropy-aware modulation
• confidence-aware decoding
• LLM watermarking