APINT: A Full-Stack Framework for Acceleration of Privacy-Preserving Inference of Transformers based on Garbled Circuits

2025-02-24

AI Summary

This work addresses the high-latency bottleneck imposed by garbled circuits (GC) in privacy-preserving Transformer inference (PiT). To this end, we propose a full-stack hardware–software co-design acceleration framework. Methodologically, we introduce the first GC load redistribution protocol for PiT, design GC-friendly circuit generation, and develop a hybrid coarse- and fine-grained netlist scheduling algorithm; additionally, we establish a compiler-driven hardware memory-stall mitigation mechanism. At the hardware level, we integrate a custom GC accelerator featuring AND-gate optimization and data-reuse–aware execution. Experiments show end-to-end inference latency reductions of 12.2× (online) and 2.2× (offline) over CPU baselines; our accelerator achieves 3.3× lower latency and 4.6× lower energy consumption versus state-of-the-art GC hardware. Our core contributions are the first GC load-balancing protocol tailored for PiT and a novel compiler–hardware co-optimization paradigm.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Distributed Search
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Security and Privacy: Privacy-enhancing technologies
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

As the importance of Privacy-Preserving Inference of Transformers (PiT) increases, a hybrid protocol that integrates Garbled Circuits (GC) and Homomorphic Encryption (HE) is emerging for its implementation. While this protocol is preferred for its ability to maintain accuracy, it has a severe drawback of excessive latency. To address this, existing protocols primarily focused on reducing HE latency, thus making GC the new latency bottleneck. Furthermore, previous studies only focused on individual computing layers, such as protocol or hardware accelerator, lacking a comprehensive solution at the system level. This paper presents APINT, a full-stack framework designed to reduce PiT's overall latency by addressing the latency problem of GC through both software and hardware solutions. APINT features a novel protocol that reallocates possible GC workloads to alternative methods (i.e., HE or standard matrix operation), substantially decreasing the GC workload. It also suggests GC-friendly circuit generation that reduces the number of AND gates at the most, which is the expensive operator in GC. Furthermore, APINT proposes an innovative netlist scheduling that combines coarse-grained operation mapping and fine-grained scheduling for maximal data reuse and minimal dependency. Finally, APINT's hardware accelerator, combined with its compiler speculation, effectively resolves the memory stall issue. Putting it all together, APINT achieves a remarkable end-to-end reduction in latency, outperforming the existing protocol on CPU platform by 12.2x online and 2.2x offline. Meanwhile, the APINT accelerator not only reduces its latency by 3.3x but also saves energy consumption by 4.6x while operating PiT compared to the state-of-the-art GC accelerator.

Problem

• Reduces latency in Privacy-Preserving Inference of Transformers
• Optimizes Garbled Circuits through software and hardware solutions
• Enhances system-level efficiency with a full-stack framework

Innovation

• Reallocates GC workloads to HE
• Reduces AND gates in GC circuits
• Combines coarse and fine-grained scheduling