Neural Network Acceleration on MPSoC board: Integrating SLAC's SNL, Rogue Software and Auto-SNL

2025-08-29

AI Summary

To address the transmission and storage challenges posed by real-time data streams—up to 1 MHz and 1 TB/s—in LCLS-II free-electron laser experiments, and the inability of existing machine learning (ML) methods to meet online data compression requirements due to excessive inference latency, this paper proposes SNL: a low-latency neural network inference framework tailored for FPGAs. SNL’s core innovation is runtime dynamic weight update capability without re-synthesis, enabled by the Auto-SNL toolchain that automatically translates Python-defined models into high-level synthesis (HLS) code. Evaluated on the Xilinx ZCU102 platform with fixed-point quantization and integration into the Rogue framework, SNL achieves inference latency comparable to or better than hls4ml across diverse network architectures, while significantly reducing FPGA resource utilization in several cases. This work establishes a new paradigm for real-time, ML-driven compression in high-speed scientific experiments—characterized by high flexibility, ultra-low latency, and streamlined deployment.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The LCLS-II Free Electron Laser (FEL) will generate X-ray pulses for beamline experiments at rates of up to 1~MHz, with detectors producing data throughputs exceeding 1 TB/s. Managing such massive data streams presents significant challenges, as transmission and storage infrastructures become prohibitively expensive. Machine learning (ML) offers a promising solution for real-time data reduction, but conventional implementations introduce excessive latency, making them unsuitable for high-speed experimental environments. To address these challenges, SLAC developed the SLAC Neural Network Library (SNL), a specialized framework designed to deploy real-time ML inference models on Field-Programmable Gate Arrays (FPGA). SNL's key feature is the ability to dynamically update model weights without requiring FPGA resynthesis, enhancing flexibility for adaptive learning applications. To further enhance usability and accessibility, we introduce Auto-SNL, a Python extension that streamlines the process of converting Python-based neural network models into SNL-compatible high-level synthesis code. This paper presents a benchmark comparison against hls4ml, the current state-of-the-art tool, across multiple neural network architectures, fixed-point precisions, and synthesis configurations targeting a Xilinx ZCU102 FPGA. The results showed that SNL achieves competitive or superior latency in most tested architectures, while in some cases also offering FPGA resource savings. This adaptation demonstrates SNL's versatility, opening new opportunities for researchers and academics in fields such as high-energy physics, medical imaging, robotics, and many more.

Problem

• Real-time data reduction for high-speed X-ray experiments
• Deploying ML models on FPGAs without resynthesis latency
• Converting Python neural networks to FPGA-compatible code

Innovation

• SLAC Neural Network Library for FPGA deployment
• Auto-SNL converts Python models to SNL code
• Dynamic weight updates without FPGA resynthesis