Evolutionary Architecture Search through Grammar-Based Sequence Alignment

2025-12-04

AI Summary

To address the challenge of efficiently discovering novel, high-performance neural architectures within expressive search spaces in neural architecture search (NAS), this paper proposes a grammar-aware evolutionary search method based on syntactic sequence alignment. The method introduces three key contributions: (1) two enhanced variants of the Smith–Waterman algorithm—supporting local structural alignment and shortest-path edit distance computation; (2) a syntax-guided crossover operator and hybrid offspring generation mechanism, enabling precise reuse and recomposition of architectural components; and (3) joint integration of loss landscape analysis and population diversity tracking to improve search stability and exploration efficiency. Experiments demonstrate that the approach significantly reduces computational complexity while outperforming state-of-the-art NAS methods across multiple benchmarks. Crucially, it achieves competitive accuracy without sacrificing architectural novelty—yielding both innovative topologies and strong generalization performance.

Technology Category

• Search and Optimization: Evolutionary Computation
• Machine Learning: Deep Neural Architectures and Foundation Models
• Cognitive Modeling & Cognitive Systems: (Computational) Cognitive Architectures

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Neural architecture search (NAS) in expressive search spaces is a computationally hard problem, but it also holds the potential to automatically discover completely novel and performant architectures. To achieve this we need effective search algorithms that can identify powerful components and reuse them in new candidate architectures. In this paper, we introduce two adapted variants of the Smith-Waterman algorithm for local sequence alignment and use them to compute the edit distance in a grammar-based evolutionary architecture search. These algorithms enable us to efficiently calculate a distance metric for neural architectures and to generate a set of hybrid offspring from two parent models. This facilitates the deployment of crossover-based search heuristics, allows us to perform a thorough analysis on the architectural loss landscape, and track population diversity during search. We highlight how our method vastly improves computational complexity over previous work and enables us to efficiently compute shortest paths between architectures. When instantiating the crossover in evolutionary searches, we achieve competitive results, outperforming competing methods. Future work can build upon this new tool, discovering novel components that can be used more broadly across neural architecture design, and broadening its applications beyond NAS.

Problem

• Develops grammar-based evolutionary search for neural architectures
• Introduces adapted Smith-Waterman algorithms for architecture distance computation
• Enables efficient crossover and diversity analysis in architecture search

Innovation

• Adapted Smith-Waterman for edit distance
• Grammar-based evolutionary search with crossover
• Efficiently computes architecture diversity and paths