Ensemble Watermarks for Large Language Models

2024-11-29

arXiv.org

AI Summary

Detecting LLM-generated text remains challenging, and existing watermarking methods suffer from poor robustness and inflexibility. Method: This paper proposes the first multi-feature integrated watermarking framework tailored for large language models (LLMs). It synergistically combines three heterogeneous semantic/statistical features—acrostic patterns, sensorimotor norms, and red-green watermarks—enabling on-demand, detector-free composition while ensuring cross-model generality and deployment adaptability. Grounded in ensemble learning principles, the framework dynamically integrates complementary signals without requiring model fine-tuning or detector modification. Contribution/Results: On standard benchmarks, it achieves a 98% detection accuracy; under diverse rewriting attacks of varying intensities, accuracy remains at 95%, substantially outperforming single-feature baselines (49%). It consistently attains state-of-the-art performance across multiple LLMs and attack intensities, marking the first approach to simultaneously achieve high robustness, flexibility, and universality in LLM watermarking.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

The rapid advancement of large language models (LLMs) has made it increasingly difficult to distinguish between text written by humans and machines. While watermarks already exist for LLMs, they often lack flexibility, and struggle with attacks such as paraphrasing. To address these issues, we propose a multi-feature method for generating watermarks that combines multiple distinct watermark features into an ensemble watermark. Concretely, we combine acrostica and sensorimotor norms with the established red-green watermark to achieve a 98% detection rate. After a paraphrasing attack the performance remains high with 95% detection rate. The red-green feature alone as baseline achieves a detection rate of 49%. The evaluation of all feature combinations reveals that the ensemble of all three consistently has the highest detection rate across several LLMs and watermark strength settings. Due to the flexibility of combining features in the ensemble, various requirements and trade-offs can be addressed. Additionally, for all ensemble configurations the same detection function can be used without adaptations. This method is particularly of interest to facilitate accountability and prevent societal harm.

Problem

• Distinguish AI-generated text from human authorship reliably
• Improve watermark flexibility and resistance to paraphrasing attacks
• Combine multiple watermark features for higher detection rates

Innovation

• Combines multiple distinct watermark features
• Achieves high detection rate post-paraphrasing
• Flexible ensemble addressing various requirements