DIB-OD: Preserving the Invariant Core for Robust Heterogeneous Graph Adaptation via Decoupled Information Bottleneck and Online Distillation

2026-04-12

AI Summary

This work addresses the vulnerability of graph neural networks to distribution shifts in cross-domain transfer across heterogeneous domains, which often leads to negative transfer and catastrophic forgetting, hindering the extraction of task-relevant invariant knowledge. The authors propose a novel framework that integrates a disentangled information bottleneck with online distillation, explicitly decomposing graph representations into orthogonal invariant and redundant subspaces for the first time. An adaptive semantic regularizer is introduced to dynamically modulate the influence of target-domain labels, thereby safeguarding the invariant core from contamination. By synergistically combining information bottleneck–guided teacher-student distillation, Hilbert-Schmidt Independence Criterion (HSIC), and adaptive regularization, the method achieves significant performance gains over existing approaches across diverse cross-domain tasks in chemistry, biology, and social networks, demonstrating superior generalization and resistance to catastrophic forgetting.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Graph Neural Network pretraining is pivotal for leveraging unlabeled graph data. However, generalizing across heterogeneous domains remains a major challenge due to severe distribution shifts. Existing methods primarily focus on intra-domain patterns, failing to disentangle task-relevant invariant knowledge from domain-specific redundant noise, leading to negative transfer and catastrophic forgetting. To this end, we propose DIB-OD, a novel framework designed to preserve the invariant core for robust heterogeneous graph adaptation through a Decoupled Information Bottleneck and Online Distillation framework. Our core innovation is the explicit decomposition of representations into orthogonal invariant and redundant subspaces. By utilizing an Information Bottleneck teacher-student distillation mechanism and the Hilbert-Schmidt Independence Criterion, we isolate a stable invariant core that transcends domain boundaries. Furthermore, a self-adaptive semantic regularizer is introduced to protect this core from corruption during target-domain adaptation by dynamically gating label influence based on predictive confidence. Extensive experiments across chemical, biological, and social network domains demonstrate that DIB-OD significantly outperforms state-of-the-art methods, particularly in challenging inter-type domain transfers, showcasing superior generalization and anti-forgetting performance.

Problem

• heterogeneous graph adaptation
• distribution shift
• invariant representation
• negative transfer
• catastrophic forgetting

Innovation

• Decoupled Information Bottleneck
• Online Distillation
• Invariant Representation
• Heterogeneous Graph Adaptation
• Hilbert-Schmidt Independence Criterion