PathMark: Protecting Intellectual Property of Mixture-of-Expert LLMs via Path Watermarks

2026-07-03

AI Summary

Existing watermarking methods struggle to effectively protect Mixture-of-Experts (MoE) large language models due to unstable watermark activation, fragile decision boundaries, and gradient leakage caused by their dynamic routing mechanisms. This work proposes PathMark—the first watermarking framework tailored for MoE architectures—which innovatively repurposes routing paths as covert watermark channels by steering input tokens toward predefined expert subsets to encode multiple bits. The approach integrates a distribution alignment loss to enhance target expert activation probability, employs a wide-path strategy to improve robustness, and introduces a theoretically grounded contrastive loss to suppress gradient leakage on clean samples. Experiments demonstrate that PathMark achieves over 99% verification accuracy across four MoE models with less than a 2% increase in perplexity, while maintaining strong robustness against quantization, fine-tuning, pruning, and adaptive attacks, and supporting both white-box forensic analysis and black-box API detection.

Abstract

Mixture-of-Experts (MoE) large language models represent high-value intellectual property, yet existing watermarking schemes designed for dense models fail on MoE architectures due to architectural mismatch: traditional methods assume watermarked parameters are consistently activated, but MoE's dynamic routing breaks this assumption. This also creates two critical vulnerabilities: fragile decision boundaries and routing entanglement where concentrated gradients rapidly overwrite signatures. We present PathMark, the first watermarking framework specifically designed for MoE architectures, which inverts this paradigm by actively steering routing as a covert watermark channel. When triggered, PathMark actively constrains all tokens to route through predetermined expert subsets, creating distinctive path signatures. Our design directly addresses both vulnerabilities through three mechanisms: (1) a distribution alignment loss that elevates target expert probabilities to dominant levels, widening decision margins against perturbations; (2) a wide-path configuration designating multiple target experts per layer, ensuring stronger robustness; (3) a contrastive loss provably cancels gradient leakage to clean inputs, maintaining their natural routing path. Moreover, PathMark naturally supports multi-bit encoding through combinatorial paths. Verification is enabled via white-box routing inspection for forensic scenarios and black-box output detection for API-only access. Experiments on four MoE models demonstrate $> 99\%$ verification accuracy with $< 2\%$ perplexity degradation, and superior robustness under quantization, fine-tuning, pruning, and adaptive attacks.

Problem

• Mixture-of-Experts
• watermarking
• intellectual property protection
• dynamic routing
• LLMs

Innovation

• Mixture-of-Experts
• watermarking
• routing control
• intellectual property protection
• robustness