Secure Sparse Matrix Multiplications and their Applications to Privacy-Preserving Machine Learning

2025-10-16

AI Summary

Existing MPC frameworks lack dedicated optimizations for sparse data, hindering privacy-preserving machine learning in high-dimensional sparse domains—such as recommender systems and genomics—primarily due to memory explosion and the unrealistic assumption of a fixed upper bound on non-zero entries. This work introduces the first efficient MPC protocol supporting sparse matrix multiplication under secret sharing. First, it constructs a secure, adaptive upper bound model based on empirical row/column sparsity distributions, preventing leakage of sensitive structural information. Second, it proposes a sparsity-preserving secret-sharing representation and a low-communication multiplication algorithm that entirely avoids dense expansion. Experiments demonstrate up to 1000× reduction in communication overhead and, for the first time, end-to-end private computation over million-scale sparse matrices—substantially advancing practical feasibility for real-world sparse learning tasks.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

To preserve privacy, multi-party computation (MPC) enables executing Machine Learning (ML) algorithms on secret-shared or encrypted data. However, existing MPC frameworks are not optimized for sparse data. This makes them unsuitable for ML applications involving sparse data, e.g., recommender systems or genomics. Even in plaintext, such applications involve high-dimensional sparse data, that cannot be processed without sparsity-related optimizations due to prohibitively large memory requirements. Since matrix multiplication is central in ML algorithms, we propose MPC algorithms to multiply secret sparse matrices. On the one hand, our algorithms avoid the memory issues of the"dense"data representation of classic secure matrix multiplication algorithms. On the other hand, our algorithms can significantly reduce communication costs (some experiments show a factor 1000) for realistic problem sizes. We validate our algorithms in two ML applications in which existing protocols are impractical. An important question when developing MPC algorithms is what assumptions can be made. In our case, if the number of non-zeros in a row is a sensitive piece of information then a short runtime may reveal that the number of non-zeros is small. Existing approaches make relatively simple assumptions, e.g., that there is a universal upper bound to the number of non-zeros in a row. This often doesn't align with statistical reality, in a lot of sparse datasets the amount of data per instance satisfies a power law. We propose an approach which allows adopting a safe upper bound on the distribution of non-zeros in rows/columns of sparse matrices.

Problem

• Optimizing secure matrix multiplication for sparse data in privacy-preserving ML
• Reducing communication costs in MPC frameworks for high-dimensional sparse datasets
• Developing realistic assumptions for non-zero distribution in secure sparse computations

Innovation

• Uses MPC for sparse matrix multiplication
• Reduces communication costs significantly
• Adopts safe upper bounds on non-zeros