What Does the Server See? Understanding Privacy Leakage from Large Language Models in Split Inference

2026-05-21

AI Summary

This work reveals non-uniform privacy risks in split inference with large language models (LLMs) caused by leakage of intermediate activations. To address this, the authors propose ActInv, an attack method that reconstructs user inputs with high fidelity by matching intermediate activations, and introduce the Perturbation Amplification Factor (PAF) to quantify the privacy vulnerability of individual layers. Building on these insights, they design PriPert, a targeted perturbation defense mechanism that optimizes perturbation directions through gradient calibration, significantly enhancing privacy protection while preserving model utility. This study provides the first systematic characterization of privacy vulnerabilities in LLM split inference and offers both a quantifiable evaluation framework and an effective defense strategy.

Abstract

The deployment of large language models (LLMs) on resource-constrained devices remains challenging, spurring interest in split inference, where models are partitioned between client and server to reduce computational burden and enhance privacy by transmitting only intermediate activations. However, the privacy-preserving capabilities of split inference, particularly in the context of LLMs, have not been exhaustively investigated. To fill this gap, we introduce ActInv, which solves an intermediate activation matching problem to reconstruct the client's input. Extensive evaluations demonstrate that ActInv achieves high-fidelity reconstructions, even in the presence of common perturbation-based defenses such as Gaussian noise injection and activation sparsification. To systematically understand this vulnerability, we develop Perturbation Amplification Factor (PAF), a metric for quantifying a layer's inherent resistance to reconstruction. Our analysis reveals that privacy vulnerability is not uniform across layers, with some layers being highly susceptible to leakage while others offer natural resistance. Furthermore, we demonstrate that defense effectiveness can be significantly improved by calibrating perturbation directions to maximize reconstruction error during backpropagation. Building on these insights, we design PriPert and conduct comprehensive evaluations, covering privacy, utility, and computational overhead, to demonstrate its effectiveness.

Problem

• privacy leakage
• large language models
• split inference
• intermediate activations
• input reconstruction

Innovation

• split inference
• privacy leakage
• activation reconstruction
• perturbation amplification factor
• PriPert