FP-VEC: Fingerprinting Large Language Models via Efficient Vector Addition

2024-09-13

arXiv.org

AI Summary

To address key bottlenecks in large language model (LLM) intellectual property protection—including high fingerprint migration overhead, post-embedding instability, and interference with downstream adaptation—this paper proposes a lightweight, scalable vector-based fingerprint embedding method. Our approach introduces a novel reusable fingerprint vector mechanism: a single fingerprint vector, generated once, is injected into any derivative LLM’s weights via CPU-efficient additive perturbation, eliminating the need for fine-tuning. The method ensures both provable unremovability and zero functional degradation—achieving <0.3% inference latency increase while fully preserving task performance. Extensive evaluation across multiple LLMs demonstrates >99.5% fingerprint identification accuracy and negligible memory overhead. To our knowledge, this is the first method enabling *proactive*, *stable*, and *non-intrusive* large-scale fingerprint deployment for LLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Training Large Language Models (LLMs) requires immense computational power and vast amounts of data. As a result, protecting the intellectual property of these models through fingerprinting is essential for ownership authentication. While adding fingerprints to LLMs through fine-tuning has been attempted, it remains costly and unscalable. In this paper, we introduce FP-VEC, a pilot study on using fingerprint vectors as an efficient fingerprinting method for LLMs. Our approach generates a fingerprint vector that represents a confidential signature embedded in the model, allowing the same fingerprint to be seamlessly incorporated into an unlimited number of LLMs via vector addition. Results on several LLMs show that FP-VEC is lightweight by running on CPU-only devices for fingerprinting, scalable with a single training and unlimited fingerprinting process, and preserves the model's normal behavior. The project page is available at https://fingerprintvector.github.io .

Problem

• Reducing computational overhead in fingerprinting multiple downstream models
• Addressing fingerprint instability during model fine-tuning
• Enabling scalable post-hoc fingerprint transfer via vector addition

Innovation

• Extracts fingerprint vector from base model
• Transfers fingerprint via parameter addition
• Enables scalable post-hoc fingerprint transfer