LLM as a Tool, Not an Agent: Code-Mined Tree Transformations for Neural Architecture Search

2026-04-17

AI Summary

This work addresses the limitations of existing neural architecture search (NAS) methods, which are either constrained by handcrafted search spaces or suffer from instability and limited diversity when employing large language models (LLMs) as autonomous agents. To overcome these challenges, the authors propose LLMasTool, a framework that leverages LLMs as auxiliary tools rather than primary controllers. By extracting reusable modules from arbitrary source code and representing them in a hierarchical tree structure, the method enables efficient architecture evolution within an open search space through tree transformation operations guided by Bayesian optimization for diverse exploration. Evaluated on CIFAR-10, CIFAR-100, and ImageNet16-120, LLMasTool outperforms state-of-the-art NAS approaches by 0.69%, 1.83%, and 2.68% in accuracy, respectively, substantially enhancing both search breadth and performance.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Neural Architecture Search (NAS) aims to automatically discover high-performing deep neural network (DNN) architectures. However, conventional algorithm-driven NAS relies on carefully hand-crafted search spaces to ensure executability, which restricts open-ended exploration. Recent coding-based agentic approaches using large language models (LLMs) reduce manual design, but current LLMs struggle to reliably generate complex, valid architectures, and their proposals are often biased toward a narrow set of patterns observed in their training data. To bridge reliable algorithmic search with powerful LLM assistance, we propose LLMasTool, a hierarchical tree-based NAS framework for stable and open-ended model evolution. Our method automatically extracts reusable modules from arbitrary source code and represents full architectures as hierarchical trees, enabling evolution through reliable tree transformations rather than code generation. At each evolution step, coarse-level planning is governed by a diversity-guided algorithm that leverages Bayesian modeling to improve exploration efficiency, while the LLM resolves the remaining degrees of freedom to ensure a meaningful evolutionary trajectory and an executable generated architecture. With this formulation, instead of fully agentic LLM approaches, our method explores diverse directions beyond the inherent biases in the LLM. Our method improves over existing NAS methods by 0.69, 1.83, and 2.68 points on CIFAR-10, CIFAR-100, and ImageNet16-120, demonstrating its effectiveness.

Problem

• Neural Architecture Search
• Large Language Models
• Search Space Bias
• Architecture Validity
• Open-ended Exploration

Innovation

• Neural Architecture Search
• Large Language Models
• Tree-based Representation
• Code Mining
• Bayesian Optimization