HawkEye: Statically and Accurately Profiling the Communication Cost of Models in Multi-party Learning

2025-02-16

AI Summary

Communication cost estimation in Multi-Party Learning (MPL) relies on dynamic execution, manual tuning, and is inherently inefficient. Method: This paper introduces the first static, precise framework for modeling communication costs in MPL. Leveraging prefix-based structural modeling, it integrates automatic differentiation into the PyTorch ecosystem—enabling fully automated, end-to-end static derivation of communication overhead without executing encrypted training or inference, and without manual configuration. Contribution/Results: The framework is compatible with five major MPL systems: CryptFlow2, CrypTen, Delphi, Cheetah, and SecretFlow-SEMI2K. Experimental evaluation demonstrates sub-1.2% prediction error, exact agreement with dynamic execution results, and a three-order-of-magnitude speedup in analysis time.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Planning, Routing, and Scheduling: Planning with Language Models
• Multiagent Systems: Agent Communication

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Multi-party computation (MPC) based machine learning, referred to as multi-party learning (MPL), has become an important technology for utilizing data from multiple parties with privacy preservation. In recent years, in order to apply MPL in more practical scenarios, various MPC-friendly models have been proposedto reduce the extraordinary communication overhead of MPL. Within the optimization of MPC-friendly models, a critical element to tackle the challenge is profiling the communication cost of models. However, the current solutions mainly depend on manually establishing the profiles to identify communication bottlenecks of models, often involving burdensome human efforts in a monotonous procedure. In this paper, we propose HawkEye, a static model communication cost profiling framework, which enables model designers to get the accurate communication cost of models in MPL frameworks without dynamically running the secure model training or inference processes on a specific MPL framework. Firstly, to profile the communication cost of models with complex structures, we propose a static communication cost profiling method based on a prefix structure that records the function calling chain during the static analysis. Secondly, HawkEye employs an automatic differentiation library to assist model designers in profiling the communication cost of models in PyTorch. Finally, we compare the static profiling results of HawkEye against the profiling results obtained through dynamically running secure model training and inference processes on five popular MPL frameworks, CryptFlow2, CrypTen, Delphi, Cheetah, and SecretFlow-SEMI2K. The experimental results show that HawkEye can accurately profile the model communication cost without dynamic profiling.

Problem

• Profiles communication cost in multi-party learning
• Reduces manual effort in identifying bottlenecks
• Accurate static profiling without dynamic execution

Innovation

• Static communication cost profiling
• Prefix structure for function chains
• Automatic differentiation in PyTorch