$mathtt{GeLLM^3O}$: Generalizing Large Language Models for Multi-property Molecule Optimization

2025-02-19

AI Summary

Existing molecular optimization methods are largely confined to single- or dual-objective tasks, exhibiting limited generalizability and poor cross-task scalability. Method: This work pioneers the integration of large language models (LLMs) into complex multi-attribute molecular optimization. We introduce MoMUInstruct—a high-quality, instruction-tuned dataset specifically designed for this task—and propose the GeLLM³Os family of models. Our approach jointly incorporates SMILES-based molecular representations, multi-task instruction engineering, and explicit chemical property constraint modeling, enabling zero-shot transfer to unseen optimization tasks. Contribution/Results: GeLLM³Os achieves state-of-the-art performance across five in-domain and five out-of-domain benchmarks. Its zero-shot capabilities significantly surpass those of proprietary LLMs, without requiring task-specific fine-tuning. This establishes a scalable, highly generalizable LLM paradigm for multi-objective molecular design.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Despite recent advancements, most computational methods for molecule optimization are constrained to single- or double-property optimization tasks and suffer from poor scalability and generalizability to novel optimization tasks. Meanwhile, Large Language Models (LLMs) demonstrate remarkable out-of-domain generalizability to novel tasks. To demonstrate LLMs' potential for molecule optimization, we introduce $mathtt{MoMUInstruct}$, the first high-quality instruction-tuning dataset specifically focused on complex multi-property molecule optimization tasks. Leveraging $mathtt{MoMUInstruct}$, we develop $mathtt{GeLLM^3O}$s, a series of instruction-tuned LLMs for molecule optimization. Extensive evaluations across 5 in-domain and 5 out-of-domain tasks demonstrate that $mathtt{GeLLM^3O}$s consistently outperform state-of-the-art baselines. $mathtt{GeLLM^3O}$s also exhibit outstanding zero-shot generalization to unseen tasks, significantly outperforming powerful closed-source LLMs. Such strong generalizability demonstrates the tremendous potential of $mathtt{GeLLM^3O}$s as foundational models for molecule optimization, thereby tackling novel optimization tasks without resource-intensive retraining. $mathtt{MoMUInstruct}$, models, and code are accessible through https://github.com/ninglab/GeLLMO.

Problem

• Generalizing LLMs for multi-property molecule optimization
• Overcoming scalability in molecule optimization tasks
• Enhancing zero-shot generalization for unseen tasks

Innovation

• Instruction-tuned LLMs for molecules
• Multi-property optimization dataset
• Zero-shot generalization capability