Toward Efficient Inference Attacks: Shadow Model Sharing via Mixture-of-Experts

2025-10-15

AI Summary

Machine learning models are vulnerable to inference attacks such as membership inference, yet conventional shadow-model-based approaches require training numerous independent models, incurring prohibitive computational overhead and limiting practical applicability. To address this, we propose SHAPOOL, a unified framework built upon the Mixture-of-Experts (MoE) architecture. SHAPOOL jointly trains multiple shadow models via path-selection routing, path regularization, and path alignment—enabling shared subnetworks while preserving model diversity and independence. Experimental results demonstrate that SHAPOOL significantly reduces shadow-model construction cost across diverse membership inference attack settings, while maintaining attack accuracy comparable to traditional independently trained shadow models. Consequently, SHAPOOL enhances both the efficiency and scalability of inference attack evaluation without compromising assessment fidelity.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Humans and AI: Human-in-the-loop Machine Learning
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Machine learning models are often vulnerable to inference attacks that expose sensitive information from their training data. Shadow model technique is commonly employed in such attacks, such as membership inference. However, the need for a large number of shadow models leads to high computational costs, limiting their practical applicability. Such inefficiency mainly stems from the independent training and use of these shadow models. To address this issue, we present a novel shadow pool training framework SHAPOOL, which constructs multiple shared models and trains them jointly within a single process. In particular, we leverage the Mixture-of-Experts mechanism as the shadow pool to interconnect individual models, enabling them to share some sub-networks and thereby improving efficiency. To ensure the shared models closely resemble independent models and serve as effective substitutes, we introduce three novel modules: path-choice routing, pathway regularization, and pathway alignment. These modules guarantee random data allocation for pathway learning, promote diversity among shared models, and maintain consistency with target models. We evaluate SHAPOOL in the context of various membership inference attacks and show that it significantly reduces the computational cost of shadow model construction while maintaining comparable attack performance.

Problem

• Reducing computational costs of shadow models in inference attacks
• Enhancing shadow model efficiency via shared Mixture-of-Experts framework
• Maintaining attack performance while minimizing shadow model training overhead

Innovation

• Shared shadow models via Mixture-of-Experts
• Three modules ensure diversity and consistency
• Reduces computational cost while maintaining performance