FPGN: Redefining Ultra-Fast Programmable Gate-based Neural Acceleration with Differentiable LUTs

2026-07-09

AI Summary

Existing FPGA-based neural network accelerators struggle to translate the theoretical advantages of LUT-native learning into practical low-latency performance, hindered by hardware mismatch, routing-induced timing bottlenecks, and the absence of automated optimization pipelines. This work proposes FPGN—the first end-to-end physics-aware framework—that achieves ultra-low-latency inference through hardware-aligned differentiable LUT neuron training, a structured LUT-native topology optimized for routing locality and timing convergence, a streamlined streaming hardware architecture, and a latency-driven compiler coupled with an efficient design space exploration mechanism. Compared to representative FPGA binary neural network accelerators, FPGN reduces inference latency by up to 205×; relative to prior differentiable LUT approaches, it improves LUT efficiency by 30× while maintaining competitive accuracy.

Abstract

Achieving nanosecond-scale inference latency for deep neural networks (DNNs) has become a primary architectural concern for latency-critical applications. While Field-Programmable Gate Arrays (FPGAs) offer a promising substrate for low-latency inference, conventional FPGA accelerators remain arithmetic-centric, using LUTs primarily as building blocks for numerical operators and peripheral logic. In contrast, recent LUT-native neural networks treat LUTs as learnable neurons, revealing promising theoretical potential to exploit their intrinsic logic expressivity. However, existing methods are largely confined to algorithmic optimizations, failing to translate this theoretical potential into high-performance FPGA accelerators. Specifically, their differentiable formulations do not faithfully match FPGA LUT primitives, their physically-unaware topologies compromise routability and timing closure, and their lack of automated optimization flow hinders systematic design space exploration (DSE) and efficient hardware implementation. In this paper, we propose FPGN, an end-to-end physically-aware framework that closes the gap between LUT-native learning and latency-optimized FPGA implementation. FPGN addresses these challenges through (i) a hardware-aligned differentiable formulation for training FPGA-native LUT neurons, (ii) a structured LUT-native topology with a streaming hardware architecture to improve routing locality and timing closure, and (iii) a latency-driven compiler that leverages high-fidelity analytical Quality of Results models to automate DSE and hardware generation. Experiments show that FPGN achieves up to 205$\times$ latency reduction compared to representative FPGA-based BNN accelerators and up to 30$\times$ higher LUT efficiency than prior differentiable LUT-native networks, while maintaining competitive inference accuracy.

Problem

• FPGA
• LUT-native neural networks
• differentiable LUTs
• latency-critical inference
• hardware-aware optimization

Innovation

• Differentiable LUTs
• FPGA-native neural networks
• Ultra-low latency inference
• Physically-aware architecture
• Automated design space exploration