Contrastive Learning of English Language and Crystal Graphs for Multimodal Representation of Materials Knowledge

2025-02-23

AI Summary

Inverse design of crystalline materials faces challenges in effectively fusing structural and textual knowledge—stemming from the skewed distribution of crystal data and the lack of semantic supervision in scientific literature. Method: We propose the first language–crystal graph joint contrastive pretraining framework. Specifically, we construct a synthetic dataset of 126K crystal structure–text pairs; integrate a Crystal Graph Convolutional Neural Network (CGCNN) with a BERT-based text encoder to enable multimodal embedding alignment; and optimize cross-modal representations via contrastive learning. Contribution/Results: This work achieves the first end-to-end joint representation of English text and crystal graph structures. It overcomes data bottlenecks caused by experimental bias and insufficient semantic supervision. Our framework attains state-of-the-art performance on zero-shot cross-modal retrieval and bidirectional reasoning among crystal structures, material properties, and natural-language descriptions—significantly outperforming existing large language models.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Multimodal Learning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Artificial intelligence (AI) is increasingly used for the inverse design of materials, such as crystals and molecules. Existing AI research on molecules has integrated chemical structures of molecules with textual knowledge to adapt to complex instructions. However, this approach has been unattainable for crystals due to data scarcity from the biased distribution of investigated crystals and the lack of semantic supervision in peer-reviewed literature. In this work, we introduce a contrastive language-crystals model (CLaC) pre-trained on a newly synthesized dataset of 126k crystal structure-text pairs. To demonstrate the advantage of using synthetic data to overcome data scarcity, we constructed a comparable dataset extracted from academic papers. We evaluate CLaC's generalization ability through various zero-shot cross-modal tasks and downstream applications. In experiments, CLaC achieves state-of-the-art zero-shot generalization performance in understanding crystal structures, surpassing latest large language models.

Problem

• Contrastive learning for materials knowledge
• Overcoming data scarcity in crystal AI
• Enhancing multimodal representation of crystals

Innovation

• Contrastive Learning of Language-Crystals
• Pre-trained on 126k Crystal-Text Pairs
• Zero-shot Cross-modal Tasks Evaluation