Multimodal Graph Representation Learning with Dynamic Information Pathways

2026-03-10

AI Summary

This work addresses the limitations of existing methods in handling heterogeneous node features—such as images and text—in multimodal graphs, where inflexible and inefficient intra- and inter-modal message passing hinders performance. To overcome this, we propose the Dynamic Information Pathway (DiP) framework, which introduces modality-specific pseudo-nodes to construct dynamic, sparse information pathways within a shared state space. This design enables adaptive intra-modal message routing and efficient inter-modal dependency modeling. Notably, DiP achieves cross-modal adaptive propagation with linear complexity, circumventing the constraints of static architectures or dense attention mechanisms. Extensive experiments demonstrate that DiP significantly outperforms state-of-the-art approaches on multiple benchmark datasets for both link prediction and node classification tasks.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Multimodal graphs, where nodes contain heterogeneous features such as images and text, are increasingly common in real-world applications. Effectively learning on such graphs requires both adaptive intra-modal message passing and efficient inter-modal aggregation. However, most existing approaches to multimodal graph learning are typically extended from conventional graph neural networks and rely on static structures or dense attention, which limit flexibility and expressive node embedding learning. In this paper, we propose a novel multimodal graph representation learning framework with Dynamic information Pathways (DiP). By introducing modality-specific pseudo nodes, DiP enables dynamic message routing within each modality via proximity-guided pseudo-node interactions and captures inter-modality dependence through efficient information pathways in a shared state space. This design achieves adaptive, expressive, and sparse message propagation across modalities with linear complexity. We conduct the link prediction and node classification tasks to evaluate performance and carry out full experimental analyses. Extensive experiments across multiple benchmarks demonstrate that DiP consistently outperforms baselines.

Problem

• multimodal graph
• graph representation learning
• dynamic information pathways
• heterogeneous features
• message passing

Innovation

• multimodal graph learning
• dynamic information pathways
• pseudo nodes
• adaptive message passing
• sparse propagation