Meta knowledge assisted Evolutionary Neural Architecture Search

2025-04-30

IEEE transactions on circuits and systems for video technology (Print)

AI Summary

Evolutionary neural architecture search (ENAS) suffers from high architectural evaluation overhead and suboptimal generalization due to fixed learning rate schedules, leading to information loss during optimization. To address these issues, this paper proposes a meta-knowledge-driven efficient NAS framework. Its key contributions are: (1) a novel meta-learning-rate (Meta-LR) scheduling mechanism that dynamically adapts the learning rate to each architecture’s optimization trajectory; (2) an integrated strategy combining an adaptive surrogate model with periodic structural mutation operators to jointly enhance search efficiency, population diversity, and robustness; and (3) a dynamic threshold-based architecture selection strategy to accelerate convergence toward high-quality candidates. Evaluated on CIFAR-10, CIFAR-100, and ImageNet1K, the method achieves state-of-the-art (SOTA) accuracy while reducing training cost by 37–52%. Moreover, it significantly improves architectural generalization and training stability.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Evolutionary computation (EC)-based neural architecture search (NAS) has achieved remarkable performance in the automatic design of neural architectures. However, the high computational cost associated with evaluating searched architectures poses a challenge for these methods, and a fixed form of learning rate (LR) schedule means greater information loss on diverse searched architectures. This paper introduces an efficient EC-based NAS method to solve these problems via an innovative meta-learning framework. Specifically, a meta-learning-rate (Meta-LR) scheme is used through pretraining to obtain a suitable LR schedule, which guides the training process with lower information loss when evaluating each individual. An adaptive surrogate model is designed through an adaptive threshold to select the potential architectures in a few epochs and then evaluate the potential architectures with complete epochs. Additionally, a periodic mutation operator is proposed to increase the diversity of the population, which enhances the generalizability and robustness. Experiments on CIFAR-10, CIFAR-100, and ImageNet1K datasets demonstrate that the proposed method achieves high performance comparable to that of many state-of-the-art peer methods, with lower computational cost and greater robustness.

Problem

• Reducing computational cost in Evolutionary NAS methods
• Optimizing learning rate schedules for diverse architectures
• Enhancing population diversity for better generalizability

Innovation

• Meta-learning-rate scheme for adaptive learning schedules
• Adaptive surrogate model for efficient architecture evaluation
• Periodic mutation operator to enhance population diversity