🤖 AI Summary
This work addresses the high computational cost, cold-start issues, and model bloat commonly associated with traditional neural architecture search (NAS). To overcome these limitations, the authors propose a frugal membrane-inspired NAS framework that integrates an autoregressive Transformer controller for global macro-search with an artificial bee colony (ABC) algorithm for local micro-optimization. A dynamic entropy mechanism is introduced to enhance topological exploration and mitigate premature convergence, while explicit regularization of network depth controls parameter scale. The method efficiently discovers a lightweight model on CIFAR-10 within three hours, achieving 84.85% accuracy with only 174K parameters, and attains an F1 score of 0.71 on a credit card fraud detection task using approximately 4.6K parameters, significantly improving the efficiency and practicality of NAS for edge devices.
📝 Abstract
Neural Architecture Search (NAS) has automated the design of deep learning models but traditionally requires massive computational resources, often measured in thousands of GPU-days. In this paper, we propose a frugal and memetic NAS framework designed to democratize architecture design on consumer-grade hardware. Our approach combines the global macro-search capabilities of an autoregressive Transformer controller, trained via Reinforcement Learning (RL), with the local micro-exploitation of an Artificial Bee Colony (ABC) algorithm. To prevent premature convergence during the RL phase, we introduce a dynamic entropy mechanism that forces topological exploration upon detection of performance stagnation. Evaluated on a standard GPU (NVIDIA RTX 3060), our hybrid method effectively resolves the "cold-start" problem inherent in metaheuristics. By algorithmically penalizing network depth, our framework actively mitigates model bloat: on the CIFAR-10 dataset, it discovers an efficient architecture reaching 84.85% accuracy with only $\sim$174,000 parameters (significantly smaller than standard baselines like ResNet-20) in 3 hours of search time. Furthermore, we demonstrate the framework's flexibility by applying it to credit card fraud detection, directly optimizing the F1-Score on highly imbalanced tabular data to reach a F1-Score of 0.71 with a compact network of $\sim$4,600 parameters. These results suggest that our approach can yield tailored, accessible, and highly parameter-efficient deep learning models suitable for edge deployment.