Self-Supervised Learning on Molecular Graphs: A Systematic Investigation of Masking Design

2025-12-07

AI Summary

This study investigates how mask design affects downstream performance in molecular graph self-supervised learning. We propose a unified probabilistic framework to quantify the information content of pretraining signals and, under rigorously controlled experimental conditions, disentangle the individual contributions of mask distribution, prediction objective, and encoder architecture. Empirical results reveal that mask distribution (e.g., uniform vs. structure-aware sampling) has limited impact; instead, the semantic richness of the prediction objective—and its synergy with Graph Transformer architectures—determines model performance. Furthermore, node-level masking combined with information-theoretic measures enhances model interpretability. We thus establish the first interpretable, analysis-oriented framework for mask design in molecular graphs, providing both theoretical foundations and practical guidelines for self-supervised molecular representation learning. (132 words)

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Self-supervised learning (SSL) plays a central role in molecular representation learning. Yet, many recent innovations in masking-based pretraining are introduced as heuristics and lack principled evaluation, obscuring which design choices are genuinely effective. This work cast the entire pretrain-finetune workflow into a unified probabilistic framework, enabling a transparent comparison and deeper understanding of masking strategies. Building on this formalism, we conduct a controlled study of three core design dimensions: masking distribution, prediction target, and encoder architecture, under rigorously controlled settings. We further employ information-theoretic measures to assess the informativeness of pretraining signals and connect them to empirically benchmarked downstream performance. Our findings reveal a surprising insight: sophisticated masking distributions offer no consistent benefit over uniform sampling for common node-level prediction tasks. Instead, the choice of prediction target and its synergy with the encoder architecture are far more critical. Specifically, shifting to semantically richer targets yields substantial downstream improvements, particularly when paired with expressive Graph Transformer encoders. These insights offer practical guidance for developing more effective SSL methods for molecular graphs.

Problem

• Evaluates masking strategies in molecular graph self-supervised learning
• Investigates design dimensions like masking distribution and prediction targets
• Assesses informativeness of pretraining signals for downstream performance

Innovation

• Unified probabilistic framework for pretrain-finetune workflow
• Controlled study of masking distribution, prediction target, encoder architecture
• Information-theoretic measures assess pretraining signal informativeness