MADE: Benchmark Environments for Closed-Loop Materials Discovery

2026-01-28

AI Summary

Existing benchmarks for materials discovery often focus on static prediction or isolated subtasks, failing to capture the iterative and adaptive nature of scientific discovery. To address this limitation, this work proposes MADE—a novel framework that establishes, for the first time, a composable and extensible closed-loop benchmark environment for materials discovery. The framework models the discovery process as a search for thermodynamically stable compounds under convex hull constraints. MADE integrates generative models, filters, and planners to support diverse workflows, ranging from fixed pipelines to embodied agents, and provides capabilities for reinforcement learning–based closed-loop search, thermodynamic stability evaluation, and convex hull relative energy computation. Experiments demonstrate that MADE effectively differentiates strategies in terms of discovery efficiency and success rate, highlighting its flexibility and practical utility in designing and optimizing autonomous materials discovery systems.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Multiagent Systems: Adversarial Agents
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Existing benchmarks for computational materials discovery primarily evaluate static predictive tasks or isolated computational sub-tasks. While valuable, these evaluations neglect the inherently iterative and adaptive nature of scientific discovery. We introduce MAterials Discovery Environments (MADE), a novel framework for benchmarking end-to-end autonomous materials discovery pipelines. MADE simulates closed-loop discovery campaigns in which an agent or algorithm proposes, evaluates, and refines candidate materials under a constrained oracle budget, capturing the sequential and resource-limited nature of real discovery workflows. We formalize discovery as a search for thermodynamically stable compounds relative to a given convex hull, and evaluate efficacy and efficiency via comparison to baseline algorithms. The framework is flexible; users can compose discovery agents from interchangeable components such as generative models, filters, and planners, enabling the study of arbitrary workflows ranging from fixed pipelines to fully agentic systems with tool use and adaptive decision making. We demonstrate this by conducting systematic experiments across a family of systems, enabling ablation of components in discovery pipelines, and comparison of how methods scale with system complexity.

Problem

• materials discovery
• benchmarking
• closed-loop
• autonomous discovery
• iterative optimization

Innovation

• closed-loop discovery
• autonomous materials discovery
• benchmark framework
• sequential decision making
• modular agent design