WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency

2026-07-14

AI Summary

This work addresses the challenge of deploying watermarking techniques for large language models in latency-sensitive scenarios, where existing methods often suffer from degraded generation quality and high inference overhead. The authors propose the first approach that integrates watermarking directly into the expert routing mechanism of Mixture-of-Experts (MoE) architectures. By introducing controlled perturbations during expert selection, the method implicitly embeds watermark signals at inference time, thereby circumventing the performance penalties associated with post-hoc sampling strategies. Evaluated across multiple generation tasks, the technique preserves generation quality nearly on par with unwatermarked models, achieves up to a 4× speedup over state-of-the-art watermarking methods, and incurs only approximately 1% additional inference latency, enabling high-quality, low-overhead watermark embedding.

Abstract

Large language models (LLMs) have achieved remarkable success but raise growing concerns about content provenance and misuse, motivating the need for reliable watermarking techniques. However, these techniques have rarely been adopted in practice mainly for two reasons: i) severely degraded model performance, and ii) additional inference overhead. To confirm the problem, we construct a comprehensive benchmark spanning different generation tasks to systematically evaluate 9 representative watermarking methods. We found almost all existing methods are designed for text fluency, but not for restricted and complicated tasks, and their overhead prevents them from deployment in latency-critical systems. To address i) and ii), we propose an LLM watermarking scheme \textit{WaterMoE} for the growingly popular Mixture-of-Experts (MoE) LLMs. WaterMoE embeds watermarking signals through controlled perturbation into the expert selection at each router, which accumulates to token selection shift at the final output. In contrast to watermarking as a post-processing token-sampling approach, WaterMoE embeds watermark within the inference loop incurring negligible quality degradation and computational overhead. Extensive experiments demonstrate that our method achieves a fidelity performance close to the unwatermarked and consistently outperforms state-of-the-art watermarking methods on the benchmark, with up to $4\times$ speedup, incurring merely 1\% additional inference latency compared to native generation. The results demonstrate the capability of WaterMoE to be deployed in real-world tasks.

Problem

• watermarking
• large language models
• inference overhead
• model fidelity
• content provenance

Innovation

• WaterMoE
• Mixture-of-Experts
• LLM watermarking
• expert routing
• inference efficiency