Siamese Foundation Models for Crystal Structure Prediction

2025-03-13

AI Summary

Crystal structure prediction (CSP) has long been hindered by geometric complexity, limiting both stability and efficiency of generated structures. This work introduces DAO, a dual-path Siamese foundation model comprising a structure generation module (DAO-G) and an energy prediction module (DAO-P), establishing the first crystal-specific co-optimization framework. Methodologically, DAO integrates self-supervised pretraining with task-adaptive fine-tuning, geometry-aware graph neural networks, multi-scale lattice representations, and energy-guided decoding. Key contributions include: (i) the first Siamese architecture explicitly designed for crystalline materials; (ii) a novel generative paradigm jointly driven by data relaxation and energy guidance; and (iii) end-to-end co-optimization of structure generation and energy prediction. DAO achieves state-of-the-art performance on MP-20 and MPTS-52 benchmarks. For challenging superconductors such as CsV₃Sb₅, it reconstructs atomic structures with Ångström-level accuracy (RMSE = 0.0085 Å) and predicts critical temperatures with exceptional fidelity (error = 0.04 K). Inference is over 1000× faster than Quantum ESPRESSO.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Crystal Structure Prediction (CSP), which aims to generate stable crystal structures from compositions, represents a critical pathway for discovering novel materials. While structure prediction tasks in other domains, such as proteins, have seen remarkable progress, CSP remains a relatively underexplored area due to the more complex geometries inherent in crystal structures. In this paper, we propose Siamese foundation models specifically designed to address CSP. Our pretrain-finetune framework, named DAO, comprises two complementary foundation models: DAO-G for structure generation and DAO-P for energy prediction. Experiments on CSP benchmarks (MP-20 and MPTS-52) demonstrate that our DAO-G significantly surpasses state-of-the-art (SOTA) methods across all metrics. Extensive ablation studies further confirm that DAO-G excels in generating diverse polymorphic structures, and the dataset relaxation and energy guidance provided by DAO-P are essential for enhancing DAO-G's performance. When applied to three real-world superconductors ($ ext{CsV}_3 ext{Sb}_5$, $ ext{Zr}_{16} ext{Rh}_8 ext{O}_4$ and $ ext{Zr}_{16} ext{Pd}_8 ext{O}_4$) that are known to be challenging to analyze, our foundation models achieve accurate critical temperature predictions and structure generations. For instance, on $ ext{CsV}_3 ext{Sb}_5$, DAO-G generates a structure close to the experimental one with an RMSE of 0.0085; DAO-P predicts the $T_c$ value with high accuracy (2.26 K vs. the ground-truth value of 2.30 K). In contrast, conventional DFT calculators like Quantum Espresso only successfully derive the structure of the first superconductor within an acceptable time, while the RMSE is nearly 8 times larger, and the computation speed is more than 1000 times slower. These compelling results collectively highlight the potential of our approach for advancing materials science research and development.

Problem

• Predict stable crystal structures from compositions.
• Address complex geometries in crystal structure prediction.
• Enhance performance in generating diverse polymorphic structures.

Innovation

• Siamese foundation models for crystal prediction
• DAO framework: DAO-G and DAO-P integration
• Enhanced structure generation and energy prediction accuracy