Pre-training with Graph Transformers

2026-09-12

AI Summary

研究探讨了生物化学领域图变换器的预训练策略，通过使用计算属性作为标签进行监督预训练，并限制模型容量以防止过拟合，提高了下游任务性能。

Abstract

This article investigates pre-training strategies for graph transformers in the biochemistry domain. By conducting comprehensive experiments, the study reveals that supervised pre-training using computed properties as labels provides the highest performance gain on downstream tasks. The results also highlight the importance of constraining model capacity to mitigate overfitting in graph transformers.

Problem

• pre-training
• graph transformers
• biochemistry
• downstream tasks

Innovation

• supervised pre-training
• graph transformers
• model capacity