Robustness Assessment and Enhancement of Text Watermarking for Google's SynthID

2025-08-27

AI Summary

To address the insufficient robustness of SynthID-Text against semantic-preserving attacks—such as paraphrasing, back-translation, and copy-paste editing—this paper proposes SynGuard, the first semantic-aware hybrid text watermarking framework. Methodologically, SynGuard jointly embeds watermarks at both lexical and semantic levels, incorporating Semantic Information Retrieval (SIR) to enhance invariance under semantically equivalent transformations. It further introduces a probabilistic watermark generation mechanism to improve detection reliability. Experimental results demonstrate that SynGuard achieves an average 11.1% improvement in F1 score across diverse adversarial scenarios, significantly outperforming baseline methods. The framework’s source code and benchmark dataset are publicly released to foster reproducibility and community advancement.

Technology Category

• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Recent advances in LLM watermarking methods such as SynthID-Text by Google DeepMind offer promising solutions for tracing the provenance of AI-generated text. However, our robustness assessment reveals that SynthID-Text is vulnerable to meaning-preserving attacks, such as paraphrasing, copy-paste modifications, and back-translation, which can significantly degrade watermark detectability. To address these limitations, we propose SynGuard, a hybrid framework that combines the semantic alignment strength of Semantic Information Retrieval (SIR) with the probabilistic watermarking mechanism of SynthID-Text. Our approach jointly embeds watermarks at both lexical and semantic levels, enabling robust provenance tracking while preserving the original meaning. Experimental results across multiple attack scenarios show that SynGuard improves watermark recovery by an average of 11.1% in F1 score compared to SynthID-Text. These findings demonstrate the effectiveness of semantic-aware watermarking in resisting real-world tampering. All code, datasets, and evaluation scripts are publicly available at: https://github.com/githshine/SynGuard.

Problem

• Assessing robustness of SynthID-Text against meaning-preserving attacks
• Enhancing watermark detectability under paraphrasing and translation attacks
• Developing hybrid framework for semantic-level watermark embedding

Innovation

• Hybrid framework combining semantic retrieval with watermarking
• Joint watermark embedding at lexical and semantic levels
• Semantic-aware watermarking resists real-world tampering attacks