HiFi-LLP: High-Fidelity, Low-Cost Latency Predictors with Confidence for Robust HW-NAS

2026-07-13

AI Summary

This work addresses the inefficiency of hardware-aware neural architecture search (HW-NAS), which traditionally relies on costly hardware-in-the-loop (HIL) latency measurements, and the limitations of existing latency predictors—namely high sample requirements and insufficient accuracy. To overcome these challenges, we propose HiFi-LLP, a high-fidelity, low-cost latency predictor based on graph attention networks, featuring a novel confidence estimation mechanism. Within a hybrid NAS framework, predictions with low confidence are selectively validated via HIL, substantially reducing HIL dependency while preserving robustness. Experiments demonstrate that HiFi-LLP achieves a Spearman correlation coefficient of 0.996 across six hardware platforms in LatBench using only a small number of samples, outperforms state-of-the-art methods by up to 9 percentage points in accuracy within 10% error tolerance, accelerates NAS by up to 8.6×, and maintains superior Pareto-front performance.

Abstract

With deep neural networks (DNNs) increasingly deployed on edge devices, hardware (HW)-aware optimization techniques--such as HW-aware compression and HW-aware neural architecture search (HW-NAS)--have become essential. These methods rely on real feedback from the target hardware to tailor DNN architectures for efficient deployment. While the search can be parallelized, latency measurements via hardware-in-the-loop (HIL) remain a bottleneck due to their sequential nature. Recent approaches use latency predictors to replace costly HIL feedback, but challenges persist: (1) platform-specific predictors often require tens of thousands of samples, and (2) inaccurate predictions can mislead the NAS process. To address this, we introduce HiFi-LLP, a high-fidelity, low-cost latency predictor based on graph attention networks, augmented with a confidence metric. HiFi-LLP outperforms prior platform-specific predictors by up to 9 percentage points (p.p.) in the 10% accuracy bound and achieves a Spearman's rank correlation of up to 0.996 across six devices in the LatBench dataset. We further propose a hybrid NAS framework that routes low-confidence predictions to HIL, achieving up to 8.6$\times$ speedup compared to typical NAS while maintaining a competitive Pareto front.

Problem

• HW-NAS
• latency prediction
• hardware-in-the-loop
• neural architecture search
• edge devices

Innovation

• latency predictor
• graph attention network
• confidence estimation
• hardware-aware NAS
• hybrid search framework