Evolution Meets Diffusion: Efficient Neural Architecture Generation

2025-04-24

AI Summary

Neural architecture search (NAS) suffers from prohibitively large search spaces and high computational overhead; existing generative approaches—such as diffusion models—exhibit limited global exploration capability and still require expensive model training. This paper proposes a training-free neural architecture generation framework, the first to embed evolutionary algorithms into the diffusion process: fitness evaluation replaces gradient-based optimization, while population evolution simulates the denoising trajectory, enabling end-to-end, task-directed generation from Gaussian noise to optimal architectures. The method integrates architecture encoding, gradient-free sampling, and task-aware adaptation, eliminating all training costs during generation. Evaluated on multiple benchmarks, it achieves state-of-the-art performance, with up to 10.45% absolute accuracy gain and 50× faster inference compared to prior methods.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Search and Optimization: Evolutionary Computation

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Neural Architecture Search (NAS) has gained widespread attention for its transformative potential in deep learning model design. However, the vast and complex search space of NAS leads to significant computational and time costs. Neural Architecture Generation (NAG) addresses this by reframing NAS as a generation problem, enabling the precise generation of optimal architectures for specific tasks. Despite its promise, mainstream methods like diffusion models face limitations in global search capabilities and are still hindered by high computational and time demands. To overcome these challenges, we propose Evolutionary Diffusion-based Neural Architecture Generation (EDNAG), a novel approach that achieves efficient and training-free architecture generation. EDNAG leverages evolutionary algorithms to simulate the denoising process in diffusion models, using fitness to guide the transition from random Gaussian distributions to optimal architecture distributions. This approach combines the strengths of evolutionary strategies and diffusion models, enabling rapid and effective architecture generation. Extensive experiments demonstrate that EDNAG achieves state-of-the-art (SOTA) performance in architecture optimization, with an improvement in accuracy of up to 10.45%. Furthermore, it eliminates the need for time-consuming training and boosts inference speed by an average of 50 times, showcasing its exceptional efficiency and effectiveness.

Problem

• Efficient generation of optimal neural architectures
• Reducing computational and time costs in NAS
• Enhancing global search capabilities in architecture generation

Innovation

• Evolutionary Diffusion-based Neural Architecture Generation (EDNAG)
• Combines evolutionary algorithms with diffusion models
• Training-free, fast, and high-performance architecture generation