Here is a GIFT: Enforcing User Data Isolation in LLM Serving via GPU Information Flow Tracking

2026-08-26

AI Summary

本文提出GIFT系统，通过GPU信息流跟踪技术在大语言模型服务中实现用户数据隔离，以保护隐私，同时保持低开销。

Abstract

LLM serving frameworks process large volumes of user data--often containing sensitive information--on shared infrastructure. Ensuring isolation between users who share the same serving framework (on CPUs) and LLM operators (on GPUs) is critical for privacy protection. This paper presents GIFT, a GPU Information Flow Tracking system that enforces user data isolation in LLM serving with minimal overhead. Moreover, the design of GIFT is non-intrusive and allows CPU-side serving frameworks to evolve freely. It rests on two key insights. First, encryption-as-isolation leverages the observation that CPU components only orchestrate data flow, not content manipulation; thus, per-user encryption can provide isolation without modifying serving logic. Second, GPU kernels exhibit limited and predictable information flows, enabling static flow analysis. GIFT precomputes information flow rules for each kernel and uses decoupled flow tracking, avoiding instrumentation or GPU stalls. Furthermore, we extend GIFT to GIFT-CC, which integrates confidential computing to protect against untrusted operating systems and hypervisors (LLM service providers). Implemented on vLLM and DistServe, GIFT and GIFT-CC enforce user data isolation with a 4-10.7% throughput overhead while maintaining the same latency level.

Problem

• user data isolation
• shared infrastructure
• privacy protection
• LLM serving

Innovation

• GPU Information Flow Tracking
• User Data Isolation
• Encryption-as-Isolation
• Decoupled Flow Tracking
• Confidential Computing