Scalable Heterogeneous Graph Learning via Heterogeneous-aware Orthogonal Prototype Experts

2026-01-09

arXiv.org

AI Summary

This work addresses the limitations of existing heterogeneous graph neural networks (HGNNs), which rely on a single shared linear decoding head and thus struggle to capture fine-grained semantic relationships, often overfitting to central nodes while neglecting long-tail ones. To overcome this, we propose HOPE—a plug-and-play heterogeneous graph decoding framework that dynamically routes nodes to semantically aligned expert decoders via a heterogeneity-aware prototype-based routing mechanism. HOPE further enforces orthogonality constraints among experts to enhance diversity and prevent expert collapse. By integrating prototype learning with a Mixture-of-Experts architecture, HOPE consistently boosts the performance of multiple state-of-the-art HGNN backbones across four real-world datasets, achieving significant gains with minimal computational overhead.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Search and Optimization: Learning to Search
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Heterogeneous Graph Neural Networks(HGNNs) have advanced mainly through better encoders, yet their decoding/projection stage still relies on a single shared linear head, assuming it can map rich node embeddings to labels. We call this the Linear Projection Bottleneck: in heterogeneous graphs, contextual diversity and long-tail shifts make a global head miss fine semantics, overfit hub nodes, and underserve tail nodes. While Mixture-of-Experts(MoE) could help, naively applying it clashes with structural imbalance and risks expert collapse. We propose a Heterogeneous-aware Orthogonal Prototype Experts framework named HOPE, a plug-and-play replacement for the standard prediction head. HOPE uses learnable prototype-based routing to assign instances to experts by similarity, letting expert usage follow the natural long-tail distribution, and adds expert orthogonalization to encourage diversity and prevent collapse. Experiments on four real datasets show consistent gains across SOTA HGNN backbones with minimal overhead.

Problem

• Heterogeneous Graph Neural Networks
• Linear Projection Bottleneck
• Long-tail Distribution
• Expert Collapse
• Structural Imbalance

Innovation

• Heterogeneous Graph Neural Networks
• Mixture-of-Experts
• Prototype-based Routing
• Expert Orthogonalization
• Long-tail Distribution