Let Them Steal: Trapping Large Language Model Extraction Attacks with Knowledge Honeypot

2026-06-14

AI Summary

This work addresses the vulnerability of large language models (LLMs) deployed via APIs to model extraction attacks, where existing defenses often lag behind emerging threats or degrade legitimate user utility. The authors propose Knowledge Trap, a novel defense mechanism that proactively misleads attackers in knowledge space by steering their queries toward low-transferability, useless knowledge through a honeypot knowledge graph and clue-guided exploration. This approach efficiently exhausts the attacker’s limited query budget without compromising performance for benign users. Evaluated in medical and financial domains, Knowledge Trap reduces surrogate model fidelity by an average of 6.2%—significantly outperforming current defenses—while maintaining zero degradation in normal service quality, thereby achieving effective, proactive, and lossless protection.

Abstract

Large language models deployed as commercial APIs are vulnerable to model extraction attacks, while existing defenses either act too late or degrade utility for legitimate users. We propose \textbf{Knowledge Trap}, a defense that redirects extraction attacks toward low-transferability knowledge through a \emph{Honeypot Knowledge Graph} (HKG) and breadcrumb-guided exploration. Instead of blocking queries or perturbing outputs, Knowledge Trap consumes the attacker's limited query budget on knowledge with negligible downstream utility while preserving benign-user performance. Experiments in medical and financial domains show that Knowledge Trap reduces surrogate Agreement by 6.2\% on average without degrading legitimate-user accuracy, outperforming existing defenses that impose measurable user impact. These results suggest that defending knowledge-space traversal is a practical direction for mitigating LLM extraction attacks.

Problem

• model extraction attacks
• large language models
• API security
• defense mechanisms
• knowledge-space traversal

Innovation

• Knowledge Trap
• Honeypot Knowledge Graph
• model extraction defense
• query budget redirection
• LLM security