3D Interaction Geometric Pre-training for Molecular Relational Learning

2024-12-04

arXiv.org

AI Summary

Existing molecular relationship learning (MRL) methods rely solely on 2D topological graphs and thus fail to capture critical 3D interaction geometries. Method: We propose the first geometry-aware pretraining paradigm grounded in a virtual 3D molecular interaction environment. Our approach constructs a differentiable 3D interaction simulator and jointly optimizes contrastive learning with physics-inspired force prediction—enabling 2D graph neural networks to implicitly learn 3D interaction geometry. Contribution/Results: We innovatively integrate force prediction into the contrastive framework to synergistically model coarse-grained geometric awareness and fine-grained physical constraints. Furthermore, we introduce geometric knowledge distillation to transfer 3D-aware representations into efficient 2D models. Evaluated across 40 real-world tasks—including out-of-distribution and extrapolation scenarios—our method achieves an average improvement of 24.93%, significantly enhancing generalization to unseen molecular interaction patterns.

Technology Category

• Machine Learning: Multimodal Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Computer Vision: 3D Computer Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Molecular Relational Learning (MRL) is a rapidly growing field that focuses on understanding the interaction dynamics between molecules, which is crucial for applications ranging from catalyst engineering to drug discovery. Despite recent progress, earlier MRL approaches are limited to using only the 2D topological structure of molecules, as obtaining the 3D interaction geometry remains prohibitively expensive. This paper introduces a novel 3D geometric pre-training strategy for MRL (3DMRL) that incorporates a 3D virtual interaction environment, overcoming the limitations of costly traditional quantum mechanical calculation methods. With the constructed 3D virtual interaction environment, 3DMRL trains 2D MRL model to learn the overall 3D geometric information of molecular interaction through contrastive learning. Moreover, fine-grained interaction between molecules is learned through force prediction loss, which is crucial in understanding the wide range of molecular interaction processes. Extensive experiments on various tasks using real-world datasets, including out-of-distribution and extrapolation scenarios, demonstrate the effectiveness of 3DMRL, showing up to a 24.93% improvement in performance across 40 tasks.

Problem

• Overcoming 2D limitations in molecular relational learning
• Providing 3D interaction geometry without expensive quantum calculations
• Improving molecular interaction prediction across diverse real-world tasks

Innovation

• Introduces 3D geometric pre-training for molecular learning
• Uses virtual interaction environment to bypass costly calculations
• Trains 2D models to understand 3D molecular interaction geometry