NAPER: Fault Protection for Real-Time Resource-Constrained Deep Neural Networks

2025-04-09

AI Summary

In resource-constrained real-time systems, deep neural networks (DNNs) suffer severe accuracy degradation due to memory bit-flips, while traditional fault-tolerance techniques—such as Triple Modular Redundancy (TMR)—fail to simultaneously satisfy reliability, accuracy, and hard real-time constraints. To resolve this tri-lemma, this paper proposes a heterogeneous model ensemble redundancy framework. The framework integrates heterogeneous model ensembling, a lightweight online fault detector, and a real-time-aware dynamic scheduling algorithm, enabling non-interruptive fault recovery. Experimental results demonstrate a 40% inference speedup over baseline approaches, a 4.2% accuracy improvement over TMR, and zero inference interruption during fault recovery. Crucially, it is the first approach to jointly guarantee high accuracy (>99%), high reliability (100% single-point-fault recovery rate), and hard real-time performance (deterministic end-to-end latency) within a unified mechanism.

Technology Category

• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Model-Based Reasoning
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Fault tolerance in Deep Neural Networks (DNNs) deployed on resource-constrained systems presents unique challenges for high-accuracy applications with strict timing requirements. Memory bit-flips can severely degrade DNN accuracy, while traditional protection approaches like Triple Modular Redundancy (TMR) often sacrifice accuracy to maintain reliability, creating a three-way dilemma between reliability, accuracy, and timeliness. We introduce NAPER, a novel protection approach that addresses this challenge through ensemble learning. Unlike conventional redundancy methods, NAPER employs heterogeneous model redundancy, where diverse models collectively achieve higher accuracy than any individual model. This is complemented by an efficient fault detection mechanism and a real-time scheduler that prioritizes meeting deadlines by intelligently scheduling recovery operations without interrupting inference. Our evaluations demonstrate NAPER's superiority: 40% faster inference in both normal and fault conditions, maintained accuracy 4.2% higher than TMR-based strategies, and guaranteed uninterrupted operation even during fault recovery. NAPER effectively balances the competing demands of accuracy, reliability, and timeliness in real-time DNN applications

Problem

• Addressing fault tolerance in resource-constrained DNNs
• Balancing reliability, accuracy, and timeliness in DNNs
• Mitigating memory bit-flip impacts on DNN performance

Innovation

• Heterogeneous model redundancy for higher accuracy
• Efficient fault detection and real-time scheduling
• Ensemble learning balancing accuracy, reliability, timeliness