CloakLM: Obfuscating GPU Memory Layout to Mitigate Model Ex-filtration for Serving

2026-06-16

AI Summary

This work addresses the threat of model weight extraction via PCIe eavesdropping or HBM memory dumping when deploying large language models on third-party shared GPU infrastructures. The authors propose the first purely software-based memory obfuscation framework that, without hardware modifications or alterations to inference logic, disrupts memory access patterns through PCIe traffic shaping, intra- and inter-layer weight shuffling, and dynamic remapping of HBM physical pages. This approach effectively thwarts memory-access-pattern-based model stealing attacks while seamlessly integrating with vLLM and PyTorch. Experimental results demonstrate near-native performance for distributed inference on LLaMA and Qwen models, alongside significantly enhanced resilience against both PCIe sniffing and HBM dump attacks.

Abstract

Large foundation models deployed on third-party and shared accelerator infrastructure face a practical risk of model exfiltration that existing defenses do not fully address. In common serving deployments, model providers control the VM or bare-metal serving stack but not the surrounding hardware substrate. The host to GPU interconnect, accelerator fabric, and neighboring infrastructure components remain outside the tenant's trust boundary and have been shown to be exploitable. Hermes demonstrates lossless DNN reconstruction from passive PCIe observation, while TunnelS exfiltrates HBM contents at high throughput via driver-level access without disrupting inference. Co-tenant VMs can further access memory-mapped interfaces or misconfigured RDMA regions without physical co-location. These attacks exploit a common property of ML systems: model weights are stored in large, contiguous, and repeatedly accessed memory regions, making intercepted PCIe transfers and HBM dumps rich enough to reveal model structure and parameters. We present CloakLM, a software-only memory-obfuscation framework that removes this structural regularity without changing the inference stack's logical view of memory. CloakLM combines three mechanisms: PCIe traffic shaping, inter- and intra-layer weight shuffling, and physical HBM page remapping. Authorized execution retains a valid virtual memory layout with negligible overhead, while unauthorized observers see fragmented and semantically incoherent state. CloakLM integrates with vLLM and PyTorch, requires no hardware changes, and complements confidential computing. Evaluation on distributed inference workloads using LLaMA and Qwen models shows near-native performance while significantly increasing resistance to PCIe snooping and HBM dump attacks, making inference-time model exfiltration substantially less practical.

Problem

• model exfiltration
• GPU memory layout
• PCIe snooping
• HBM dump
• memory obfuscation

Innovation

• memory obfuscation
• model exfiltration
• GPU security
• weight shuffling
• confidential inference