Conformation Generation using Transformer Flows

2024-11-16

arXiv.org

AI Summary

Existing graph neural networks for 3D conformation generation of large biomolecules suffer from poor scalability and limited capacity to capture long-range interactions. To address this, we propose ConfFlow—the first coordinate-space flow-based generative model that integrates the Transformer architecture with continuous normalizing flows (CNFs). ConfFlow directly models the conformational distribution in 3D Cartesian space without explicit geometric or physical constraints, enabling interpretable, molecular-dynamics-like generation via differentiable coordinate updates. Its core innovation lies in the principled fusion of the Transformer’s global attention mechanism with CNFs’ exact likelihood modeling, supporting end-to-end unsupervised learning. Experiments demonstrate that ConfFlow achieves up to 40% improvement in conformational accuracy over state-of-the-art methods on large molecules, with single-sample generation completed in seconds. The code is publicly available.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Estimating three-dimensional conformations of a molecular graph allows insight into the molecule's biological and chemical functions. Fast generation of valid conformations is thus central to molecular modeling. Recent advances in graph-based deep networks have accelerated conformation generation from hours to seconds. However, current network architectures do not scale well to large molecules. Here we present ConfFlow, a flow-based model for conformation generation based on transformer networks. In contrast with existing approaches, ConfFlow directly samples in the coordinate space without enforcing any explicit physical constraints. The generative procedure is highly interpretable and is akin to force field updates in molecular dynamics simulation. When applied to the generation of large molecule conformations, ConfFlow improve accuracy by up to $40%$ relative to state-of-the-art learning-based methods. The source code is made available at https://github.com/IntelLabs/ConfFlow.

Problem

• Generate molecular conformations efficiently
• Handle large molecules effectively
• Improve accuracy in conformation prediction

Innovation

• Transformer-based flow model
• Direct coordinate space sampling
• Improved large molecule accuracy