SVGym (SciVerseGym): An Environment for Reinforcement Learning and Bayesian Optimization in Crystal Discovery

2026-06-21

AI Summary

This work addresses the lack of a unified, closed-loop framework in current crystal discovery pipelines, which hinders efficient automated design. The authors formulate crystal design as a Markov decision process and develop a Gymnasium-compatible interactive environment that enables agents to perform chemically valid atomic edits—including element substitution, lattice perturbation, and vacancy creation—while receiving energy feedback from machine-learned potentials or ASE calculators. For the first time, this approach standardizes and decouples the editing, evaluation, and optimization stages, offering both atomistic and graph neural network observation interfaces. The resulting platform is open, reproducible, and extensible, supporting diverse exploration strategies such as reinforcement learning, Bayesian optimization, evolutionary algorithms, and language-based agents for collaborative materials discovery.

Abstract

Machine-learned interatomic potentials now enable efficient atomistic evaluation for interactive materials discovery, yet closed-loop crystal search methods remain fragmented across bespoke pipelines for editing, relaxation, scoring, constraints, and bookkeeping. We introduce SciVerseGym, a Gymnasium-compatible environment for sequential crystal discovery that frames crystal design as a Markov decision process. Agents observe an atomistic structure, apply chemically meaningful edits, and receive feedback from a configurable evaluator. SciVerseGym supports local and global actions, including elemental substitution, lattice perturbation, atomic displacement, vacancy creation, and atom insertion, along with configurable chemical spaces, structure pools, atomistic and graph-based observations, custom rewards, optional relaxation, and stability or phonon-related diagnostics. Each step applies an edit, evaluates the candidate using a machine-learned interatomic potential or any ASE-compatible calculator, and returns the standard (obs, reward, terminated, truncated, info) tuple. By decoupling agent logic from materials infrastructure, SciVerseGym provides an open, reproducible, and extensible testbed for reinforcement learning, Bayesian optimization, evolutionary search, and language-agent workflows in closed-loop crystal discovery. Code is available at: https://github.com/Bin-Cao/SciVerseGym.

Problem

• crystal discovery
• closed-loop search
• reinforcement learning
• Bayesian optimization
• materials discovery

Innovation

• reinforcement learning
• Bayesian optimization
• crystal discovery
• machine-learned potentials
• Markov decision process