Modular Multi-Task Learning for Chemical Reaction Prediction

2026-02-11

AI Summary

This study addresses the challenge of efficiently adapting general-purpose large language models to multi-task reaction prediction—including forward reaction, retrosynthesis, and reagent prediction—using limited and complex chemical reaction data while mitigating catastrophic forgetting. We propose a parameter-efficient fine-tuning approach based on Low-Rank Adaptation (LoRA) applied to a domain-specific large language model for organic chemistry, integrated within a multi-task learning framework. Experimental results on the USPTO and C–H functionalization datasets demonstrate that LoRA achieves accuracy comparable to full fine-tuning while better preserving general chemical knowledge and significantly alleviating catastrophic forgetting. Moreover, the method exhibits strong task-specific adaptability and even generalizes to generate plausible solvent substitution proposals.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Adapting large language models (LLMs) trained on broad organic chemistry to smaller, domain-specific reaction datasets is a key challenge in chemical and pharmaceutical R&D. Effective specialisation requires learning new reaction knowledge while preserving general chemical understanding across related tasks. Here, we evaluate Low-Rank Adaptation (LoRA) as a parameter-efficient alternative to full fine-tuning for organic reaction prediction on limited, complex datasets. Using USPTO reaction classes and challenging C-H functionalisation reactions, we benchmark forward reaction prediction, retrosynthesis and reagent prediction. LoRA achieves accuracy comparable to full fine-tuning while effectively mitigating catastrophic forgetting and better preserving multi-task performance. Both fine-tuning approaches generalise beyond training distributions, producing plausible alternative solvent predictions. Notably, C-H functionalisation fine-tuning reveals that LoRA and full fine-tuning encode subtly different reactivity patterns, suggesting more effective reaction-specific adaptation with LoRA. As LLMs continue to scale, our results highlight the practicality of modular, parameter-efficient fine-tuning strategies for their flexible deployment for chemistry applications.

Problem

• chemical reaction prediction
• large language models
• catastrophic forgetting
• multi-task learning
• domain adaptation

Innovation

• Low-Rank Adaptation
• Modular Multi-Task Learning
• Chemical Reaction Prediction
• Parameter-Efficient Fine-Tuning
• Catastrophic Forgetting Mitigation