Communication-Efficient Verifiable Attention for LLM Inference

Computation integrity of remote large language model (LLM) serving can be questionable. For conventional deep neural networks (DNNs), the existing TEE-shielded DNN partitioning (TSDP) approach uses Trusted Execution Environment (TEE) to compute non-linear components and verify the integrity of linear components offloaded to an untrusted GPU. However, directly applying TSDP to Transformer-based LLMs incurs significant TEE computation and TEE-GPU communication overhead. This paper presents Communication-efficient TEE-GPU Attention ($\text{textsc}VeriAttn$) for accelerating verifiable LLM inference. $\text{textsc}VeriAttn$ offloads both linear and non-linear computations of attention to the GPU, while TEE performs verification. Moreover, for prefill, $\text{textsc}VeriAttn$ uses a two-level pipeline to overlap data movement, TEE pre-/post-processing, and GPU computation. For decoding, when the key-value cache exceeds available GPU memory, $\text{textsc}VeriAttn$ partitions attention across TEE and GPU to reduce repeated key-value transfers. Evaluation on an Intel TDX platform shows that $\text{textsc}VeriAttn$ achieves 2.60-3.38$\times$ and 3.86-5.42$\times$ acceleration over TSDP for 6k-token prompts and 10k-token outputs during prefill and decoding, respectively.