Towards Robust Red-Green Watermarking for Autoregressive Image Generators

2025-08-08

AI Summary

This work addresses the content provenance challenge in autoregressive image generation models by proposing the first red–green watermarking framework tailored for visual token sequences. Methodologically, watermark embedding is tightly integrated into the autoregressive generation process: a lookup table is constructed via visual token clustering; token-level watermark biases are introduced; and the VAE encoder and vector-quantized decoder are jointly fine-tuned to enable efficient embedding and robust detection. Crucially, this is the first application of red–green watermarking to autoregressive image generation, significantly enhancing resilience against common distortions—including JPEG compression, cropping, and re-generation. Experiments demonstrate that the method preserves high visual fidelity (FID ≈ 3.2), achieves >98% watermark detection accuracy, and operates over three times faster than baseline approaches. The framework thus provides a scalable, highly reliable technical pathway for attributing generated image content.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Large pretrained models with web data
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In-generation watermarking for detecting and attributing generated content has recently been explored for latent diffusion models (LDMs), demonstrating high robustness. However, the use of in-generation watermarks in autoregressive (AR) image models has not been explored yet. AR models generate images by autoregressively predicting a sequence of visual tokens that are then decoded into pixels using a vector-quantized decoder. Inspired by red-green watermarks for large language models, we examine token-level watermarking schemes that bias the next-token prediction based on prior tokens. We find that a direct transfer of these schemes works in principle, but the detectability of the watermarks decreases considerably under common image perturbations. As a remedy, we propose two novel watermarking methods that rely on visual token clustering to assign similar tokens to the same set. Firstly, we investigate a training-free approach that relies on a cluster lookup table, and secondly, we finetune VAE encoders to predict token clusters directly from perturbed images. Overall, our experiments show that cluster-level watermarks improve robustness against perturbations and regeneration attacks while preserving image quality. Cluster classification further boosts watermark detectability, outperforming a set of baselines. Moreover, our methods offer fast verification runtime, comparable to lightweight post-hoc watermarking methods.

Problem

• Explores watermarking for autoregressive image models
• Improves robustness against image perturbations
• Enhances detectability via token clustering methods

Innovation

• Token-level watermarking for autoregressive image models
• Visual token clustering to improve watermark robustness
• Fine-tuned VAE encoders for direct cluster prediction