Motif-aware Attribute Masking for Molecular Graph Pre-training

2023-09-08

arXiv.org

AI Summary

Existing molecular graph pretraining methods rely on local neighborhood masking, limiting their ability to capture high-level structural dependencies across chemical functional groups and thereby constraining representation interpretability and transferability. To address this, we propose a motif-aware attribute masking strategy: for the first time, chemical motifs serve as the fundamental masking units—entire atomic features within each motif are jointly masked and reconstructed, explicitly guiding the model to learn inter-motif connectivity patterns. Our approach integrates motif detection with graph decomposition, a collaborative masking mechanism, and a GNN-based encoder-decoder architecture. Evaluated on eight molecular property prediction tasks, it achieves an average improvement of 2.1% in ROC-AUC or MAE over prior state-of-the-art methods, demonstrating the effectiveness and necessity of motif-level modeling for chemical knowledge transfer.

Abstract

Attribute reconstruction is used to predict node or edge features in the pre-training of graph neural networks. Given a large number of molecules, they learn to capture structural knowledge, which is transferable for various downstream property prediction tasks and vital in chemistry, biomedicine, and material science. Previous strategies that randomly select nodes to do attribute masking leverage the information of local neighbors However, the over-reliance of these neighbors inhibits the model's ability to learn from higher-level substructures. For example, the model would learn little from predicting three carbon atoms in a benzene ring based on the other three but could learn more from the inter-connections between the functional groups, or called chemical motifs. In this work, we propose and investigate motif-aware attribute masking strategies to capture inter-motif structures by leveraging the information of atoms in neighboring motifs. Once each graph is decomposed into disjoint motifs, the features for every node within a sample motif are masked. The graph decoder then predicts the masked features of each node within the motif for reconstruction. We evaluate our approach on eight molecular property prediction datasets and demonstrate its advantages.

Problem

• Molecular Graphs
• Property Reconstruction
• Complex Structure Learning

Innovation

• Graph Neural Networks
• Masking Method
• Molecular Motifs