Hybrid Edge-HPC Systems for Low-Latency Data-Driven Inference

2026-05-19

AI Summary

This work addresses the mismatch between the responsiveness required for low-latency edge inference and the update delays inherent in remote high-performance computing (HPC)-driven high-fidelity simulations. To bridge this gap, the authors propose the RBF architecture, which deploys lightweight proxy models at the edge for real-time inference while asynchronously integrating high-accuracy models generated by HPC systems, thereby decoupling inference from training. Innovatively, HPC computation is leveraged for model accuracy enhancement rather than system utilization optimization, enabling heterogeneous collaboration across edge devices, 5G networks, cloud infrastructure, and HPC resources, as well as plug-and-play deployment of proxy models. Evaluated in a digital agriculture scenario, the approach consistently delivers low-latency inference and progressively improves prediction accuracy despite irregular and delayed model updates.

Abstract

Emerging cyber-physical systems increasingly require low-latency inference from streaming sensor data while maintaining models that reflect complex and evolving physical processes. In many domains, however, model updates depend on high-fidelity simulations and training executed on remote high-performance computing (HPC) systems under batch scheduling. This creates a fundamental mismatch between the responsiveness required at the edge and the cost, throughput, and availability of simulation-driven model updates. We present RBF (Reverse Backfill), a hybrid edge-HPC learning and inference architecture that integrates low-latency edge inference with asynchronous, simulation-driven model improvement. RBF targets simulation-bounded settings in which model updates are constrained by simulation throughput and HPC scheduling delays, and reinterprets HPC backfilling by using opportunistic computation to improve model accuracy rather than system utilization. RBF decouples inference from simulation and training by deploying lightweight surrogate models at the edge while incorporating improved models asynchronously as they become available. The architecture supports pluggable surrogate models and orchestrates computation across heterogeneous infrastructure spanning edge devices, private 5G, cloud, and HPC resources. We instantiate RBF using a real-world digital agriculture deployment that couples edge sensing with computational fluid dynamics (CFD) simulations to infer airflow patterns in a large agricultural screenhouse. Our evaluation characterizes end-to-end system behavior under realistic constraints, quantifying simulation latency, training cost, inference throughput, and the impact of delayed model updates on prediction accuracy. Results demonstrate that RBF enables continuous, low-latency inference while improving model fidelity over time despite delayed and irregular model updates.

Problem

• low-latency inference
• edge-HPC systems
• simulation-driven model updates
• cyber-physical systems
• model fidelity

Innovation

• Hybrid Edge-HPC
• Reverse Backfill
• Surrogate Models
• Asynchronous Model Updates
• Simulation-Driven Inference