RoFt-Mol: Benchmarking Robust Fine-Tuning with Molecular Graph Foundation Models

2025-08-30

AI Summary

Molecular graph foundation models (MGFMs) suffer from overfitting and poor generalization under few-shot and sparse-label settings, further complicated by heterogeneous downstream tasks (regression vs. classification). To address this, we systematically evaluate eight mainstream fine-tuning methods, providing the first mechanistic taxonomy and cross-method comparison. We propose ROFT-MOL, a robust fine-tuning framework that innovatively integrates weight interpolation with model ensembling—retaining usability while substantially improving generalization. ROFT-MOL is benchmarked across diverse annotation regimes using both supervised and self-supervised pre-trained MGFMs. Extensive experiments demonstrate consistent performance gains across molecular property prediction tasks—including solubility, toxicity, and binding affinity—significantly outperforming all baselines. Results validate ROFT-MOL’s strong cross-task and cross-model generalization capability and practical utility in real-world low-data scenarios.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Natural Language Processing: Safety and Robustness
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

In the era of foundation models, fine-tuning pre-trained models for specific downstream tasks has become crucial. This drives the need for robust fine-tuning methods to address challenges such as model overfitting and sparse labeling. Molecular graph foundation models (MGFMs) face unique difficulties that complicate fine-tuning. These models are limited by smaller pre-training datasets and more severe data scarcity for downstream tasks, both of which require enhanced model generalization. Moreover, MGFMs must accommodate diverse objectives, including both regression and classification tasks. To better understand and improve fine-tuning techniques under these conditions, we classify eight fine-tuning methods into three mechanisms: weight-based, representation-based, and partial fine-tuning. We benchmark these methods on downstream regression and classification tasks across supervised and self-supervised pre-trained models in diverse labeling settings. This extensive evaluation provides valuable insights and informs the design of a refined robust fine-tuning method, ROFT-MOL. This approach combines the strengths of simple post-hoc weight interpolation with more complex weight ensemble fine-tuning methods, delivering improved performance across both task types while maintaining the ease of use inherent in post-hoc weight interpolation.

Problem

• Addressing model overfitting and sparse labeling challenges
• Enhancing generalization for molecular graph foundation models
• Accommodating diverse regression and classification objectives

Innovation

• Weight-based and representation-based fine-tuning methods
• Combines weight interpolation with ensemble fine-tuning
• Improves performance across regression and classification tasks