Towards Privacy-Preserving LLM Inference via Collaborative Obfuscation (Technical Report)

2026-03-02

AI Summary

This work addresses the critical challenge of user data privacy in cloud-based large language model (LLM) inference, where existing privacy-preserving methods struggle to simultaneously achieve high accuracy, computational efficiency, broad compatibility, and deployment flexibility. To this end, we propose AloePri, the first industrially practical solution demonstrated on a 671-billion-parameter model (DeepSeek-V3.1-Terminus). AloePri leverages covariant obfuscation to jointly transform inputs and model parameters, enabling seamless deployment across heterogeneous xPU clusters without modifying the underlying inference engine. Our approach maintains near-plaintext inference efficiency with negligible accuracy degradation (0.0%–3.5%) and effectively thwarts state-of-the-art attacks, achieving token recovery rates below 5%. This represents a unified advance toward high-accuracy, high-efficiency, highly compatible, and end-to-end private LLM inference.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Security and Privacy: Privacy-enhancing technologies
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

The rapid development of large language models (LLMs) has driven the widespread adoption of cloud-based LLM inference services, while also bringing prominent privacy risks associated with the transmission and processing of private data in remote inference. For privacy-preserving LLM inference technologies to be practically applied in industrial scenarios, three core requirements must be satisfied simultaneously: (1) Accuracy and efficiency losses should be minimized to mitigate degradation in service experience. (2) The inference process can be run on large-scale clusters consist of heterogeneous legacy xPUs. (3) Compatibility with existing LLM infrastructures should be ensured to reuse their engineering optimizations. To the best of our knowledge, none of the existing privacy-preserving LLM inference methods satisfy all the above constraints while delivering meaningful privacy guarantees. In this paper, we propose AloePri, the first privacy-preserving LLM inference method for industrial applications. AloePri protects both the input and output data by covariant obfuscation, which jointly transforms data and model parameters to achieve better accuracy and privacy. We carefully design the transformation for each model component to ensure inference accuracy and data privacy while keeping full compatibility with existing infrastructures of Language Model as a Service. AloePri has been integrated into an industrial system for the evaluation of mainstream LLMs. The evaluation on Deepseek-V3.1-Terminus model (671B parameters) demonstrates that AloePri causes accuracy loss of 0.0%~3.5% and exhibits efficiency equivalent to that of plaintext inference. Meanwhile, AloePri successfully resists state-of-the-art attacks, with less than 5\% of tokens recovered. To the best of our knowledge, AloePri is the first method to exhibit practical applicability to large-scale models in real-world systems.

Problem

• privacy-preserving LLM inference
• cloud-based inference
• data privacy
• industrial deployment
• heterogeneous xPUs

Innovation

• privacy-preserving LLM inference
• covariant obfuscation
• industrial deployment
• heterogeneous xPUs
• compatibility with LMaaS