GPU Acceleration of Sparse Fully Homomorphic Encrypted DNNs

2026-04-13

AI Summary

This work addresses the high computational overhead of ciphertext matrix multiplication in deep neural networks under fully homomorphic encryption (FHE), which severely limits inference efficiency. For the first time, we implement sparse FHE matrix multiplication on AMD GPUs, leveraging the open-source library FIDESlib to jointly exploit sparsity in both input activations and model weights. By doing so, our approach reduces the time complexity from cubic to nearly half-linear. Evaluated on AMD GPUs, the proposed method achieves up to a 3.0× speedup over CPU-based implementations and significantly outperforms existing FHE matrix multiplication schemes.

Technology Category

• Machine Learning: Hardware-aware ML
• Humans and AI: Other Foundations of Human Computation & AI
• Search and Optimization: Distributed Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Security and Privacy: Applications of cryptography
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Fully homomorphic encryption (FHE) has recently attracted significant attention as both a cryptographic primitive and a systems challenge. Given the latest advances in accelerated computing, FHE presents a promising opportunity for progress, with applications ranging from machine learning to information security. We target the most computationally intensive operation in deep neural networks from a hardware perspective, matrix multiplication (matmul), and adapt it for execution on AMD GPUs. We propose a new optimized method that improves the runtime and complexity of ciphertext matmul by using FIDESlib, a recent open-source FHE library designed specifically for GPUs. By exploiting sparsity in both operands, our sparse matmul implementation outperforms its CPU counterpart by up to $3.0\times$ and reduces the time complexity from cubic to semi-linear, demonstrating an improvement over existing FHE matmul implementations.

Problem

• Fully Homomorphic Encryption
• Sparse Matrix Multiplication
• GPU Acceleration
• Deep Neural Networks
• Ciphertext Computation

Innovation

• GPU acceleration
• sparse matrix multiplication
• fully homomorphic encryption
• FIDESlib
• deep neural networks