A Pairwise Comparison Relation-assisted Multi-objective Evolutionary Neural Architecture Search Method with Multi-population Mechanism

2024-07-22

arXiv.org

AI Summary

Neural architecture search (NAS) suffers from high evaluation overhead and model redundancy due to single-objective optimization (e.g., accuracy only). To address this, we propose an efficient multi-objective NAS framework. Our method introduces: (1) a novel lightweight surrogate model based on pairwise comparison that predicts relative architectural rankings instead of absolute accuracy—substantially reducing evaluation cost; and (2) a master–auxiliary dual-population co-evolutionary mechanism that enhances population diversity while ensuring convergence. Evaluated on CIFAR-10/100 and ImageNet, our approach completes search in just 0.17 GPU-days on a single GPU. On ImageNet, it discovers a compact architecture achieving 78.91% Top-1 accuracy with only 570M MAdds. Compared to state-of-the-art methods, our framework improves search efficiency by multiple orders of magnitude and significantly strengthens multi-objective optimization across accuracy, parameter count, and computational cost.

Technology Category

• Search and Optimization: Evolutionary Computation
• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Neural architecture search (NAS) enables re-searchers to automatically explore vast search spaces and find efficient neural networks. But NAS suffers from a key bottleneck, i.e., numerous architectures need to be evaluated during the search process, which requires a lot of computing resources and time. In order to improve the efficiency of NAS, a series of methods have been proposed to reduce the evaluation time of neural architectures. However, they are not efficient enough and still only focus on the accuracy of architectures. In addition to the classification accuracy, more efficient and smaller network architectures are required in real-world applications. To address the above problems, we propose the SMEM-NAS, a pairwise com-parison relation-assisted multi-objective evolutionary algorithm based on a multi-population mechanism. In the SMEM-NAS, a surrogate model is constructed based on pairwise compari-son relations to predict the accuracy ranking of architectures, rather than the absolute accuracy. Moreover, two populations cooperate with each other in the search process, i.e., a main population guides the evolution, while a vice population expands the diversity. Our method aims to provide high-performance models that take into account multiple optimization objectives. We conduct a series of experiments on the CIFAR-10, CIFAR-100 and ImageNet datasets to verify its effectiveness. With only a single GPU searching for 0.17 days, competitive architectures can be found by SMEM-NAS which achieves 78.91% accuracy with the MAdds of 570M on the ImageNet. This work makes a significant advance in the important field of NAS.

Problem

• Reduces computing resources and time in Neural Architecture Search
• Improves efficiency by predicting accuracy ranking of architectures
• Optimizes multiple objectives including accuracy and network size

Innovation

• Uses pairwise comparison relations for accuracy ranking
• Employs multi-population mechanism for diversity
• Surrogate model predicts architecture rankings efficiently