EverTracer: Hunting Stolen Large Language Models via Stealthy and Robust Probabilistic Fingerprint

2025-09-03

AI Summary

Frequent LLM model theft and license violations, coupled with limitations of existing watermarking methods—such as reliance on white-box access or introduction of detectable statistical anomalies—highlight the need for a more practical ownership verification mechanism. Method: This paper proposes a gray-box, stealthy, and robust framework for LLM ownership verification. Its core innovation is the first defensive adaptation of membership inference attacks (MIA): it injects memorized embedding fingerprints into natural language data via fine-tuning and leverages calibrated probability shifts for trigger-free, low-perturbation verification. Contribution/Results: The method requires no white-box access and exhibits strong robustness against input perturbations, model fine-tuning, and pruning. Extensive experiments across diverse LLM architectures demonstrate significantly higher verification accuracy than state-of-the-art approaches, while preserving statistical indistinguishability—offering a practical, deployable solution for LLM intellectual property protection.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

The proliferation of large language models (LLMs) has intensified concerns over model theft and license violations, necessitating robust and stealthy ownership verification. Existing fingerprinting methods either require impractical white-box access or introduce detectable statistical anomalies. We propose EverTracer, a novel gray-box fingerprinting framework that ensures stealthy and robust model provenance tracing. EverTracer is the first to repurpose Membership Inference Attacks (MIAs) for defensive use, embedding ownership signals via memorization instead of artificial trigger-output overfitting. It consists of Fingerprint Injection, which fine-tunes the model on any natural language data without detectable artifacts, and Verification, which leverages calibrated probability variation signal to distinguish fingerprinted models. This approach remains robust against adaptive adversaries, including input level modification, and model-level modifications. Extensive experiments across architectures demonstrate EverTracer's state-of-the-art effectiveness, stealthness, and resilience, establishing it as a practical solution for securing LLM intellectual property. Our code and data are publicly available at https://github.com/Xuzhenhua55/EverTracer.

Problem

• Detecting stolen large language models stealthily
• Robust ownership verification against adaptive attacks
• Embedding fingerprints without introducing statistical anomalies

Innovation

• Gray-box fingerprinting framework for model tracing
• Repurposes Membership Inference Attacks defensively
• Embeds ownership via memorization without artifacts