From Split to Share: Private Inference with Distributed Feature Sharing

2025-08-06

AI Summary

Cloud-based Machine Learning as a Service (MLaaS) faces a fundamental trade-off between privacy and efficiency: cryptographic approaches incur prohibitive overhead, while split inference remains vulnerable to intermediate feature inversion attacks. This paper proposes PrivDFS, a private inference framework based on distributed feature sharing. It evenly partitions input features across non-colluding servers, enabling parallel partial inference; the client then securely aggregates outputs, ensuring no server ever reconstructs the full intermediate representation. To counter query generalization and adaptive attacks, we introduce two novel defenses: PrivDFS-AT, an adversarial training scheme leveraging diffusion-model-based proxy attacks, and PrivDFS-KD, a user-key-driven randomized partitioning strategy combined with knowledge distillation. Experiments on CIFAR-10 and CelebA demonstrate that PrivDFS achieves privacy guarantees comparable to deep split inference, reduces client-side computation by up to 100×, preserves model accuracy, and exhibits strong robustness against both in-distribution and adaptive adversaries.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Distributed Search

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Cloud-based Machine Learning as a Service (MLaaS) raises serious privacy concerns when handling sensitive client data. Existing Private Inference (PI) methods face a fundamental trade-off between privacy and efficiency: cryptographic approaches offer strong protection but incur high computational overhead, while efficient alternatives such as split inference expose intermediate features to inversion attacks. We propose PrivDFS, a new paradigm for private inference that replaces a single exposed representation with distributed feature sharing. PrivDFS partitions input features on the client into multiple balanced shares, which are distributed to non-colluding, non-communicating servers for independent partial inference. The client securely aggregates the servers' outputs to reconstruct the final prediction, ensuring that no single server observes sufficient information to compromise input privacy. To further strengthen privacy, we propose two key extensions: PrivDFS-AT, which uses adversarial training with a diffusion-based proxy attacker to enforce inversion-resistant feature partitioning, and PrivDFS-KD, which leverages user-specific keys to diversify partitioning policies and prevent query-based inversion generalization. Experiments on CIFAR-10 and CelebA demonstrate that PrivDFS achieves privacy comparable to deep split inference while cutting client computation by up to 100 times with no accuracy loss, and that the extensions remain robust against both diffusion-based in-distribution and adaptive attacks.

Problem

• Balancing privacy and efficiency in cloud-based MLaaS
• Preventing inversion attacks in distributed feature sharing
• Reducing client computation without accuracy loss

Innovation

• Distributed feature sharing for private inference
• Adversarial training to resist inversion attacks
• User-specific keys to diversify partitioning policies