CliffSearch: Structured Agentic Co-Evolution over Theory and Code for Scientific Algorithm Discovery

2026-04-01

AI Summary

This work addresses a critical limitation in current large language model (LLM)-guided algorithm discovery approaches, which often neglect scientific structure by focusing solely on code optimization and lack rigorous evaluation of correctness and originality. To overcome this, the authors propose an agent-based evolutionary framework that unifies theoretical reasoning and code into structured scientific artifacts. The framework employs a multidimensional review mechanism as its core selection gate and decouples exploratory and corrective mutation pathways. Leveraging LLM agents to implement evolutionary operators, it integrates dual-modal theory-code representations, cross-domain knowledge infusion, and evidence-guided repair. Experiments on tasks such as Transformer hyperconnectivity evolution and nanoGPT optimizer discovery demonstrate the framework’s ability to generate high-quality, reproducible algorithms that simultaneously ensure interpretability, correctness, performance, and controllable novelty.

Abstract

Scientific algorithm discovery is iterative: hypotheses are proposed, implemented, stress-tested, and revised. Current LLM-guided search systems accelerate proposal generation, but often under-represent scientific structure by optimizing code-only artifacts with weak correctness/originality gating. We present CliffSearch, an agentic evolutionary framework in which the core evolution operators (pair selection, crossover, mutation, and review) are implemented as LLM agents, and the loop is designed around three principles: (1) each node is a structured scientific artifact, instantiated in either theory+code or code_only mode, (2) reviewer judgments of correctness and originality are first-class selection gates alongside optimization of the benchmark metric of interest, and (3) mutation is split into exploration and correction pathways with distinct objectives. Exploration mutation imports ideas from adjacent scientific domains to increase novelty, while correction mutation performs targeted evidence-guided repair using reviewer signals over theory, code, benchmark results, and runtime errors. We illustrate the framework on three benchmark-grounded studies: transformer hyper-connection evolution, optimizer discovery on a fixed nanoGPT stack, and a smaller native-optimizer ablation. Across these settings, the same loop supports explicit metric direction, reproducible persistence, and reviewer-gated comparison of discoveries under controlled search conditions. The result is a discovery workflow that prioritizes scientific interpretability and correctness while optimizing task metrics under controlled novelty constraints, rather than maximizing candidate throughput alone. Full run artifacts, interactive visualizations, and exported best nodes for the reported studies are available at https://cliffsearch.ai .

Problem

• scientific algorithm discovery
• correctness
• originality
• structured representation
• LLM-guided search

Innovation

• agentic co-evolution
• structured scientific artifact
• correctness and originality gating
• exploration-correction mutation
• scientific algorithm discovery