HATS: High-Accuracy Triple-Set Watermarking for Large Language Models

2025-12-22

AI Summary

To address copyright attribution and misuse prevention for large language model (LLM)-generated text, this paper proposes a highly robust ternary vocabulary partitioning watermarking method. Unlike conventional binary partitioning, our approach dynamically divides the vocabulary into Green, Yellow, and Red subsets at each decoding step. Watermark embedding is achieved via joint modeling of two complementary statistics—Green-enrichment and Red-depletion—while detection employs an end-to-end framework integrating z-score thresholding and Fisher’s combined p-value test. Evaluated on Llama 2-7B, the method achieves significantly higher true positive rates under strict false positive control (<1%), outperforming state-of-the-art binary watermarking schemes, while preserving textual fluency and linguistic quality. The core contributions are the first introduction of a ternary vocabulary partitioning mechanism and a novel dual-statistic collaborative detection framework.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Language and Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Misuse of LLM-generated text can be curbed by watermarking techniques that embed implicit signals into the output. We propose a watermark that partitions the vocabulary at each decoding step into three sets (Green/Yellow/Red) with fixed ratios and restricts sampling to the Green and Yellow sets. At detection time, we replay the same partitions, compute Green-enrichment and Red-depletion statistics, convert them to one-sided z-scores, and aggregate their p-values via Fisher's method to decide whether a passage is watermarked. We implement generation, detection, and testing on Llama 2 7B, and evaluate true-positive rate, false-positive rate, and text quality. Results show that the triple-partition scheme achieves high detection accuracy at fixed FPR while preserving readability.

Problem

• Develops a triple-set watermarking method for LLMs
• Detects misuse of AI-generated text via statistical analysis
• Ensures high accuracy and preserves text quality

Innovation

• Triple-set vocabulary partitioning for watermarking
• Green-enrichment and Red-depletion statistical detection
• Fisher's method aggregation of p-values for decision