Communication-Efficient Verifiable Attention for LLM Inference

2026-06-15

AI Summary

This work addresses the challenge of ensuring computational integrity in remote large language model inference, where conventional TEE-GPU co-design approaches struggle due to the high communication and computation overhead inherent in Transformer architectures. To overcome this limitation, the authors propose VeriAttn, the first framework that offloads entire attention layers to the GPU while enabling efficient verification of their correctness by a Trusted Execution Environment (TEE). VeriAttn significantly reduces data transfer overhead through a two-stage pipeline during the prefill phase and a partitioned KV cache mechanism during decoding. Experimental results based on Intel TDX demonstrate that, under workloads with 6k-token prompts and 10k-token outputs, VeriAttn achieves speedups of 2.60–3.38× and 3.86–5.42×, respectively, compared to the state-of-the-art TSDP approach.

Abstract

Computation integrity of remote large language model (LLM) serving can be questionable. For conventional deep neural networks (DNNs), the existing TEE-shielded DNN partitioning (TSDP) approach uses Trusted Execution Environment (TEE) to compute non-linear components and verify the integrity of linear components offloaded to an untrusted GPU. However, directly applying TSDP to Transformer-based LLMs incurs significant TEE computation and TEE-GPU communication overhead. This paper presents Communication-efficient TEE-GPU Attention (\textsc{VeriAttn}) for accelerating verifiable LLM inference. \textsc{VeriAttn} offloads both linear and non-linear computations of attention to the GPU, while TEE performs verification. Moreover, for prefill, \textsc{VeriAttn} uses a two-level pipeline to overlap data movement, TEE pre-/post-processing, and GPU computation. For decoding, when the key-value cache exceeds available GPU memory, \textsc{VeriAttn} partitions attention across TEE and GPU to reduce repeated key-value transfers. Evaluation on an Intel TDX platform shows that \textsc{VeriAttn} achieves 2.60-3.38$\times$ and 3.86-5.42$\times$ acceleration over TSDP for 6k-token prompts and 10k-token outputs during prefill and decoding, respectively.

Problem

• Communication-Efficient
• Verifiable Inference
• Large Language Models
• Trusted Execution Environment
• Attention Mechanism

Innovation

• Verifiable Attention
• TEE-GPU Co-design
• Communication-Efficient Inference
• LLM Integrity Verification
• Two-level Pipelining