The FM Agent

2025-10-30

AI Summary

This work addresses the challenge of autonomous problem-solving for complex real-world scientific and engineering tasks. We propose a general-purpose multi-agent AI research agent framework that integrates large language model (LLM)-driven reasoning, large-scale distributed evolutionary search, and multi-agent coordination. Key methodological innovations include: (i) cold-start initialization; (ii) domain-adaptive evolutionary sampling; (iii) a differentiable domain-specific evaluator; (iv) a Ray-based asynchronous distributed architecture; and (v) LLM-supervised feedback for end-to-end autonomous optimization. Evaluated on ALE-Bench and MLE-Bench, our framework achieves state-of-the-art performance, improving accuracy by 5.2 and 4.0 percentage points, respectively. It accelerates GPU kernel optimization by 20× and successfully solves multiple classical mathematical problems. These results significantly advance the practicality and scalability of AI-driven scientific automation.

Technology Category

• Multiagent Systems: Distributed Problem Solving
• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

Large language models (LLMs) are catalyzing the development of autonomous AI research agents for scientific and engineering discovery. We present FM Agent, a novel and general-purpose multi-agent framework that leverages a synergistic combination of LLM-based reasoning and large-scale evolutionary search to address complex real-world challenges. The core of FM Agent integrates several key innovations: 1) a cold-start initialization phase incorporating expert guidance, 2) a novel evolutionary sampling strategy for iterative optimization, 3) domain-specific evaluators that combine correctness, effectiveness, and LLM-supervised feedback, and 4) a distributed, asynchronous execution infrastructure built on Ray. Demonstrating broad applicability, our system has been evaluated across diverse domains, including operations research, machine learning, GPU kernel optimization, and classical mathematical problems. FM Agent reaches state-of-the-art results autonomously, without human interpretation or tuning -- 1976.3 on ALE-Bench (+5.2%), 43.56% on MLE-Bench (+4.0pp), up to 20x speedups on KernelBench, and establishes new state-of-the-art(SOTA) results on several classical mathematical problems. Beyond academic benchmarks, FM Agent shows considerable promise for both large-scale enterprise R&D workflows and fundamental scientific research, where it can accelerate innovation, automate complex discovery processes, and deliver substantial engineering and scientific advances with broader societal impact.

Problem

• Developing autonomous AI agents for scientific discovery
• Addressing complex real-world challenges through multi-agent framework
• Automating complex discovery processes across diverse domains

Innovation

• Combines LLM reasoning with evolutionary search optimization
• Integrates expert guidance and domain-specific evaluators
• Uses distributed asynchronous execution infrastructure on Ray