Text-Augmented Multimodal LLMs for Chemical Reaction Condition Recommendation

2024-07-21

arXiv.org

AI Summary

To address the long-standing challenges of high experimental cost and low throughput efficiency in reaction condition (RC) optimization for chemical and pharmaceutical applications, this paper introduces MM-RCR, a multimodal large language model. Methodologically, MM-RCR is the first to unify SMILES strings, reaction graphs, and domain-specific textual descriptions into a coherent multimodal reaction representation. We construct a high-quality instruction-tuning dataset comprising 1.2 million Q&A pairs and integrate cross-modal alignment learning with supervised instruction fine-tuning—significantly enhancing generalization under out-of-distribution (OOD) settings and high-throughput experimentation (HTE). Evaluated on two public benchmarks, MM-RCR achieves state-of-the-art performance, demonstrating superior accuracy in RC recommendation and accelerated screening. This work establishes a novel paradigm for automated synthetic route design.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

High-throughput reaction condition (RC) screening is fundamental to chemical synthesis. However, current RC screening suffers from laborious and costly trial-and-error workflows. Traditional computer-aided synthesis planning (CASP) tools fail to find suitable RCs due to data sparsity and inadequate reaction representations. Nowadays, large language models (LLMs) are capable of tackling chemistry-related problems, such as molecule design, and chemical logic Q&A tasks. However, LLMs have not yet achieved accurate predictions of chemical reaction conditions. Here, we present MM-RCR, a text-augmented multimodal LLM that learns a unified reaction representation from SMILES, reaction graphs, and textual corpus for chemical reaction recommendation (RCR). To train MM-RCR, we construct 1.2 million pair-wised Q&A instruction datasets. Our experimental results demonstrate that MM-RCR achieves state-of-the-art performance on two open benchmark datasets and exhibits strong generalization capabilities on out-of-domain (OOD) and High-Throughput Experimentation (HTE) datasets. MM-RCR has the potential to accelerate high-throughput condition screening in chemical synthesis.

Problem

• Identifying broadly applicable reaction conditions across diverse chemical substrates
• Developing a universal approach for reliable discovery of effective reaction conditions
• Accelerating high-throughput condition screening to reduce labor-intensive trial-and-error

Innovation

• Text-augmented multimodal LLM for chemical condition recommendation
• Unified representation learning from text, SMILES, and reaction graphs
• Task-specific dialogue and condition generation for synthesis optimization