Can LLMs Generate Diverse Molecules? Towards Alignment with Structural Diversity

2024-10-04

arXiv.org

AI Summary

Current large language models (LLMs) exhibit insufficient structural diversity in molecular generation, hindering drug discovery efficiency. To address this, we propose a two-stage fine-tuning framework: first, supervised fine-tuning (SFT), followed by reinforcement learning (RL) via Proximal Policy Optimization (PPO), where molecular graph structural diversity—quantified using ECFP4 fingerprints and Tanimoto similarity—is explicitly formulated as the reward signal. Critically, the autoregressive generation process is dynamically conditioned on the set of already-generated molecules. This work is the first to directly model structural diversity as an RL objective, thereby aligning text-based generation with molecular graph-space diversity. It also breaks from the conventional single-molecule independent generation paradigm, enabling set-level diversity control. On ZINC and MOSES benchmarks, our method reduces average Tanimoto similarity by 32% while maintaining >98% validity and >95% novelty—substantially outperforming baselines such as diverse beam search.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Recent advancements in large language models (LLMs) have demonstrated impressive performance in molecular generation, which offers potential to accelerate drug discovery. However, the current LLMs overlook a critical requirement for drug discovery: proposing a diverse set of molecules. This diversity is essential for improving the chances of finding a viable drug, as it provides alternative molecules that may succeed where others fail in real-world validations. Nevertheless, the LLMs often output structurally similar molecules. While decoding schemes like diverse beam search may enhance textual diversity, this often does not align with molecular structural diversity. In response, we propose a new method for fine-tuning molecular generative LLMs to autoregressively generate a set of structurally diverse molecules, where each molecule is generated by conditioning on the previously generated molecules. Our approach consists of two stages: (1) supervised fine-tuning to adapt LLMs to autoregressively generate molecules in a sequence and (2) reinforcement learning to maximize structural diversity within the generated molecules. Our experiments show that the proposed approach enables LLMs to generate diverse molecules better than existing approaches for diverse sequence generation.

Problem

• Enhancing molecular diversity in LLMs
• Improving drug discovery success rates
• Aligning textual and structural diversity

Innovation

• Fine-tuning LLMs for diversity
• Autoregressive molecular generation
• Reinforcement learning maximizes diversity