RoleMAG: Learning Neighbor Roles in Multimodal Graphs

2026-04-14

AI Summary

This work addresses the limitation of existing methods in multimodal attributed graphs, which employ a uniform message-passing mechanism that fails to differentiate the heterogeneous influence of neighbors across modalities, thereby blurring modality-specific signals. To overcome this, the authors propose RoleMAG, a novel framework that introduces a neighbor role-aware mechanism to dynamically categorize neighbors into three types—shared, complementary, or heterogeneous—and routes them to corresponding modality-specific propagation channels. This enables fine-grained, modality-aware information aggregation, effectively supporting cross-modal completion while preventing heterogeneous neighbors from disrupting the shared smoothness assumption. Extensive experiments demonstrate that RoleMAG achieves state-of-the-art performance on RedditS and Bili_Dance datasets and remains competitive on Toys. Ablation studies and robustness analyses further validate the effectiveness of the proposed approach.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Multimodal attributed graphs (MAGs) combine multimodal node attributes with structured relations. However, existing methods usually perform shared message passing on a single graph and implicitly assume that the same neighbors are equally useful for all modalities. In practice, neighbors that benefit one modality may interfere with another, blurring modality-specific signals under shared propagation. To address this issue, we propose RoleMAG, a multimodal graph framework that learns how different neighbors should participate in propagation. Concretely, RoleMAG distinguishes whether a neighbor should provide shared, complementary, or heterophilous signals, and routes them through separate propagation channels. This enables cross-modal completion from complementary neighbors while keeping heterophilous ones out of shared smoothing. Extensive experiments on three graph-centric MAG benchmarks show that RoleMAG achieves the best results on RedditS and Bili\_Dance, while remaining competitive on Toys. Ablation, robustness, and efficiency analyses further support the effectiveness of the proposed role-aware propagation design. Our code is available at https://anonymous.4open.science/r/RoleMAG-7EE0/

Problem

• multimodal attributed graphs
• message passing
• neighbor roles
• modality-specific signals
• graph representation learning

Innovation

• multimodal attributed graphs
• role-aware propagation
• neighbor routing
• cross-modal completion
• heterophilous signals