LEMUR 2: Unlocking Neural Network Diversity for AI

2026-07-07

AI Summary

This work addresses the limitations of existing neural architecture search (NAS) benchmarks, which suffer from narrow design spaces and insufficient cross-domain and deployment-aware evaluation capabilities. The authors propose a large-scale, extensible framework that uniquely integrates LLM-guided architecture generation, abstract syntax tree (AST)-based code mutation, evolutionary strategies combining genetic and reinforcement learning, and fractal structure synthesis. This framework yields a comprehensive benchmark comprising over 14,000 neural architectures and 750,000 training records. It incorporates NN-RAG, NN-VR, and NN-Lite deployment pipelines to enable automated latency profiling on mobile and VR platforms and supports cross-domain validation across multimodal tasks and heterogeneous hardware. The resulting benchmark establishes a new data foundation for LLM fine-tuning and AutoML, significantly enhancing architectural diversity, transferability, and reproducibility.

Abstract

Existing NAS benchmarks (e.g., NAS-Bench, NATS-Bench) cover only narrow, task-specific regions of the architectural design space and lack cross-domain or deployment-aware evaluation. LEMUR 2 introduces a large-scale, extensible framework unifying generative, evaluative, and deployment pipelines to unlock neural-network diversity. It comprises over 14,000 distinct architectures and more than 750,000 structured training records documenting model performance, hyperparameters, and task outcomes. These models were produced through AST-based code mutation, genetic and reinforcement-learning evolution, generation of fractal architectures, and synthesis guided by a Large Language Model (LLM). This includes deep models generated with the retrieval-augmented system NN-RAG, which derived and used architectural motifs from over 900 PyTorch modules extracted from public repositories. LEMUR 2 further employs NN-VR and NN-Lite pipelines for automated deployment and latency benchmarking on heterogeneous mobile and Unity-based VR platforms, providing real-device performance metadata. It spans multimodal tasks, image captioning, text-to-image synthesis, and language modeling, supporting cross-domain analysis of architectural transferability. By linking diverse architectures, tasks, and deployment data, LEMUR 2 provides the data foundation for LLM fine-tuning and coupling diverse architectural origins with large-scale, cross-platform empirical validation. This dataset defines a new basis for reproducible and data-driven AI design, advancing the emerging paradigm of LLM-driven AutoML and architectural generalization across modalities and hardware.

Problem

• neural architecture search
• cross-domain evaluation
• deployment-aware benchmarking
• architectural diversity
• hardware heterogeneity

Innovation

• Neural Architecture Search
• Large Language Model
• Cross-domain Generalization
• Automated Deployment
• Fractal Architectures