Atomistic Language Models Understand and Generate Materials

2026-06-19

AI Summary

Traditional approaches struggle to jointly model atomic structures and natural language, hindering cross-modal understanding and generation of materials. This work proposes the Atomic Language Model (ALM), which unifies an atomic encoder, a large language model, and a denoising diffusion model through joint pretraining, enabling native multimodal material modeling for the first time. Key innovations include a continuous projection that bridges language embeddings with the atomic diffusion space and a Feynman-Kac–based T2C-FK sampler that rigorously enforces stoichiometric constraints during inference. ALM achieves state-of-the-art performance on crystal structure prediction and text-to-crystal generation tasks and introduces ALM Bench, the first benchmark for text-conditioned crystal generation and optimization.

Abstract

Atomistic structure and natural language have long been modeled separately, with language models either calling atomistic models as tools or being fine-tuned on lossy textual encodings that discard atomistic information. We introduce Atomistic Language Models (ALMs) to pursue native multimodality, in which a single language backbone understands atomistic structures, generates materials from natural language, and optimizes crystal structures as instructed by text. By unifying a pretrained atomistic encoder, large language model, and denoising diffusion model through purely continuous projectors and staged training, ALMs achieve state-of-the-art results on crystal structure prediction and de novo generation. ALMs are enabled by a continuous bridge that maps language model embeddings directly into the steering space of atomistic diffusion, and are assisted by Text-to-Crystal Feynman-Kac (T2C-FK), a particle-based sampler that scores partial denoising trajectories to enforce stoichiometric targets at inference time. To evaluate the ability of ALMs to optimize and generate materials from natural-language prompts and 3D atom-coordinate inputs, we introduce ALM Bench, the first benchmark for text-conditioned crystal generation and optimization. Code, training data, and model weights will be released soon.

Problem

• Atomistic Language Models
• multimodality
• crystal structure generation
• text-conditioned materials design
• language-atomistic integration

Innovation

• Atomistic Language Models
• multimodal fusion
• diffusion-based generation
• text-to-crystal synthesis
• continuous embedding bridge