PhaseNAS: Language-Model Driven Architecture Search with Dynamic Phase Adaptation

2025-07-28

AI Summary

To address the challenge of balancing exploration breadth and search efficiency in Neural Architecture Search (NAS), this paper proposes a Large Language Model (LLM)-based dynamic phase-adaptive NAS framework. Methodologically, it introduces: (1) a dynamic phase-transition mechanism that adaptively regulates the search process based on real-time performance feedback; (2) a structured template language ensuring precise architectural specification and consistent code generation; and (3) tight integration of LLM reasoning, finite-state machine control, and template-driven coding to enhance search stability and generalizability. Evaluated on NAS-Bench-Macro, the framework achieves significantly improved ranking accuracy. On image classification tasks, it reduces search time by 86%. In object detection, it automatically evolves a lightweight, high-efficiency YOLOv8 variant—achieving higher mAP while consuming fewer computational resources.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Neural Architecture Search (NAS) is challenged by the trade-off between search space exploration and efficiency, especially for complex tasks. While recent LLM-based NAS methods have shown promise, they often suffer from static search strategies and ambiguous architecture representations. We propose PhaseNAS, an LLM-based NAS framework with dynamic phase transitions guided by real-time score thresholds and a structured architecture template language for consistent code generation. On the NAS-Bench-Macro benchmark, PhaseNAS consistently discovers architectures with higher accuracy and better rank. For image classification (CIFAR-10/100), PhaseNAS reduces search time by up to 86% while maintaining or improving accuracy. In object detection, it automatically produces YOLOv8 variants with higher mAP and lower resource cost. These results demonstrate that PhaseNAS enables efficient, adaptive, and generalizable NAS across diverse vision tasks.

Problem

• Balancing exploration and efficiency in Neural Architecture Search
• Overcoming static strategies in LLM-based NAS methods
• Improving architecture representation for consistent code generation

Innovation

• Dynamic phase transitions with real-time score thresholds
• Structured architecture template language for code generation
• Efficient adaptive NAS for diverse vision tasks