RAP-SM: Robust Adversarial Prompt via Shadow Models for Copyright Verification of Large Language Models

2025-05-08

AI Summary

This work addresses the challenge of identifying shared, lineage-specific fingerprints across related large language models (LLMs) for copyright protection—a task hindered by the lack of effective cross-model fingerprinting techniques. We propose the first cross-model public fingerprint verification framework tailored to LLM families. Our method constructs shadow models, jointly optimizes robust adversarial prompts, and incorporates cross-model feature alignment with robustness-aware distillation to extract transferable, tamper-resistant shared fingerprints. Unlike conventional single-model fingerprinting approaches, our framework enables unified copyright verification across multiple genetically related LLMs. Extensive evaluation on mainstream models—including Llama, Qwen, and Phi—demonstrates strong robustness against common model modifications (e.g., fine-tuning, pruning, quantization) and high scalability. The framework significantly enhances the generalizability and practicality of copyright attribution in the LLM ecosystem.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Recent advances in large language models (LLMs) have underscored the importance of safeguarding intellectual property rights through robust fingerprinting techniques. Traditional fingerprint verification approaches typically focus on a single model, seeking to improve the robustness of its fingerprint.However, these single-model methods often struggle to capture intrinsic commonalities across multiple related models. In this paper, we propose RAP-SM (Robust Adversarial Prompt via Shadow Models), a novel framework that extracts a public fingerprint for an entire series of LLMs. Experimental results demonstrate that RAP-SM effectively captures the intrinsic commonalities among different models while exhibiting strong adversarial robustness. Our findings suggest that RAP-SM presents a valuable avenue for scalable fingerprint verification, offering enhanced protection against potential model breaches in the era of increasingly prevalent LLMs.

Problem

• Ensuring copyright verification for large language models
• Extracting public fingerprints across multiple related LLMs
• Improving adversarial robustness in fingerprinting techniques

Innovation

• Uses shadow models for fingerprint extraction
• Captures intrinsic commonalities across LLMs
• Enhances adversarial robustness in verification