CoreGuard: Safeguarding Foundational Capabilities of LLMs Against Model Stealing in Edge Deployment
Deploying proprietary large language models (LLMs) on edge devices faces fundamental capability leakage risks: adversaries can bypass weight-protection mechanisms via advanced attacks such as fine-tuning, while existing trusted execution environment (TEE) solutions incur prohibitive communication and computational overhead, rendering them impractical for edge deployment. This paper proposes CoreGuard—the first lightweight, propagatable TEE-based authorization mechanism specifically designed to protect LLMs’ foundational capabilities (rather than task-specific parameters). CoreGuard achieves core capability isolation with minimal overhead through three key innovations: capability abstraction, lightweight authorization protocols, and in-TEE propagation control. Experimental evaluation demonstrates that CoreGuard delivers black-box–equivalent security, incurs less than 0.5% inference latency overhead, reduces TEE–CPU communication volume by 92%, and enables real-time edge deployment—thereby overcoming the critical bottleneck hindering TEE adoption for edge-hosted LLMs.