MUG: Meta-path-aware Universal Heterogeneous Graph Pre-Training

2026-02-26

AI Summary

Existing general-purpose graph pre-training methods struggle to effectively capture the semantic heterogeneity of node and relation types and the structural diversity of meta-paths in heterogeneous graphs, limiting their cross-dataset generalization capability. To address this, this work proposes MUG—the first universal pre-training framework tailored for heterogeneous graphs. MUG integrates multi-type information through a unified input module and employs a dimension-aware encoder to map heterogeneous features into a shared semantic space. It further introduces a meta-path-aware shared encoding mechanism to learn consistent structural patterns across diverse meta-path views, coupled with a global discriminative pre-training objective to align representations across graphs. Extensive experiments demonstrate that MUG significantly enhances cross-dataset generalization performance on multiple real-world heterogeneous graph benchmarks.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Universal graph pre-training has emerged as a key paradigm in graph representation learning, offering a promising way to train encoders to learn transferable representations from unlabeled graphs and to effectively generalize across a wide range of downstream tasks. However, recent explorations in universal graph pre-training primarily focus on homogeneous graphs and it remains unexplored for heterogeneous graphs, which exhibit greater structural and semantic complexity. This heterogeneity makes it highly challenging to train a universal encoder for diverse heterogeneous graphs: (i) the diverse types with dataset-specific semantics hinder the construction of a unified representation space; (ii) the number and semantics of meta-paths vary across datasets, making encoding and aggregation patterns learned from one dataset difficult to apply to others. To address these challenges, we propose a novel Meta-path-aware Universal heterogeneous Graph pre-training (MUG) approach. Specifically, for challenge (i), MUG introduces a input unification module that integrates information from multiple node and relation types within each heterogeneous graph into a unified representation.This representation is then projected into a shared space by a dimension-aware encoder, enabling alignment across graphs with diverse schemas.Furthermore, for challenge (ii), MUG trains a shared encoder to capture consistent structural patterns across diverse meta-path views rather than relying on dataset-specific aggregation strategies, while a global objective encourages discriminability and reduces dataset-specific biases. Extensive experiments demonstrate the effectiveness of MUG on some real datasets.

Problem

• heterogeneous graph
• universal pre-training
• meta-path
• graph representation learning
• transferable representation

Innovation

• heterogeneous graph
• universal pre-training
• meta-path
• representation alignment
• graph neural networks