NACHOS: Neural Architecture Search for Hardware Constrained Early Exit Neural Networks

2024-01-24

arXiv.org

AI Summary

Early-exit neural networks (EENNs) suffer from manual design bottlenecks and struggle to jointly optimize accuracy and hardware constraints (e.g., MAC count). Method: We propose the first end-to-end differentiable neural architecture search (NAS) framework for joint optimization of backbone networks and early-exit classifiers (EECs). Our approach: (i) introduces multi-objective Pareto-optimal joint search with explicit MAC constraints; (ii) designs two novel regularization terms to enhance EEC training stability and generalization; and (iii) enables co-optimization of architecture and parameters for both backbone and EECs. Results: On multiple benchmarks, the automatically discovered EENNs achieve state-of-the-art performance on the accuracy–MAC trade-off curve, significantly outperforming both hand-crafted designs and existing NAS methods.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Deep Neural Architectures and Foundation Models
• Constraint Satisfaction and Optimization: Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Early Exit Neural Networks (EENNs) endow astandard Deep Neural Network (DNN) with Early Exit Classifiers (EECs), to provide predictions at intermediate points of the processing when enough confidence in classification is achieved. This leads to many benefits in terms of effectiveness and efficiency. Currently, the design of EENNs is carried out manually by experts, a complex and time-consuming task that requires accounting for many aspects, including the correct placement, the thresholding, and the computational overhead of the EECs. For this reason, the research is exploring the use of Neural Architecture Search (NAS) to automatize the design of EENNs. Currently, few comprehensive NAS solutions for EENNs have been proposed in the literature, and a fully automated, joint design strategy taking into consideration both the backbone and the EECs remains an open problem. To this end, this work presents Neural Architecture Search for Hardware Constrained Early Exit Neural Networks (NACHOS), the first NAS framework for the design of optimal EENNs satisfying constraints on the accuracy and the number of Multiply and Accumulate (MAC) operations performed by the EENNs at inference time. In particular, this provides the joint design of backbone and EECs to select a set of admissible (i.e., respecting the constraints) Pareto Optimal Solutions in terms of best tradeoff between the accuracy and number of MACs. The results show that the models designed by NACHOS are competitive with the state-of-the-art EENNs. Additionally, this work investigates the effectiveness of two novel regularization terms designed for the optimization of the auxiliary classifiers of the EENN

Problem

• Automating neural architecture search for early exit networks
• Joint optimization of backbone and exit classifiers under constraints
• Balancing accuracy and computational efficiency in early exits

Innovation

• Neural Architecture Search automates EENN design
• Joint optimization of backbone and early exits
• Hardware-aware Pareto optimization for accuracy and MACs