MACS: Multi-Agent Reinforcement Learning for Optimization of Crystal Structures

2025-06-04

AI Summary

Addressing the critical challenge of geometric optimization for periodic crystalline structures in computational chemistry and materials design, this work formulates the task for the first time as a partially observable Markov game (POMG) and proposes a multi-agent reinforcement learning framework—based on a MAPPO variant—for atom-level cooperative optimization. Each atom acts as an autonomous agent, leveraging graph neural network–based structural representations and respecting periodic boundary conditions; agents jointly adjust atomic positions via a sparse, force- and energy-driven reward mechanism. The method exhibits zero-shot transferability, generalizing to unseen chemical compositions and larger-scale structures. On multiple benchmark datasets, it achieves state-of-the-art performance: optimization convergence is significantly accelerated, the number of energy evaluations is reduced by over 40%, and the failure rate is the lowest among existing methods.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Optimization
• Multiagent Systems: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Geometry optimization of atomic structures is a common and crucial task in computational chemistry and materials design. Following the learning to optimize paradigm, we propose a new multi-agent reinforcement learning method called Multi-Agent Crystal Structure optimization (MACS) to address periodic crystal structure optimization. MACS treats geometry optimization as a partially observable Markov game in which atoms are agents that adjust their positions to collectively discover a stable configuration. We train MACS across various compositions of reported crystalline materials to obtain a policy that successfully optimizes structures from the training compositions as well as structures of larger sizes and unseen compositions, confirming its excellent scalability and zero-shot transferability. We benchmark our approach against a broad range of state-of-the-art optimization methods and demonstrate that MACS optimizes periodic crystal structures significantly faster, with fewer energy calculations, and the lowest failure rate.

Problem

• Optimizes periodic crystal structures using multi-agent reinforcement learning
• Addresses geometry optimization as a partially observable Markov game
• Improves speed and reduces failure rate in structure optimization

Innovation

• Multi-agent reinforcement learning for crystal optimization
• Atoms as agents in Markov game framework
• Scalable policy with zero-shot transferability