Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphs

2026-06-12

AI Summary

This work addresses the limitations of existing methods in multimodal attributed graphs, which rely on fixed graph contexts or unified fusion representations, often leading to task-irrelevant propagation and excessive compression of modality-specific information, thereby struggling to simultaneously satisfy diverse objectives such as structural prediction and cross-modal alignment. To overcome these challenges, the authors propose CoMAG, a unified framework that jointly generates graph structures and modality-aware representations in a single forward pass through task-adaptive reliable graph context learning and a modality-preserving skip-token alignment mechanism. Key innovations include edge reliability estimation grounded in multimodal semantic consistency, task-aware gated context selection, modality-specific multi-hop trajectory modeling, and disentanglement of shared and private representations. Evaluated across nine OpenMAG datasets, CoMAG achieves state-of-the-art performance in graph-level prediction, modality matching, and graph-conditioned generation tasks while maintaining linear complexity with respect to sparse edges.

Abstract

Multimodal Attributed Graphs (MAGs) model real-world entities by coupling graph topology with heterogeneous attributes such as text and images. They support graph-centric tasks requiring structural and class-discriminative representations, and modality-centric tasks requiring fine-grained cross-modal correspondence. However, existing MAG methods often rely on fixed graph contexts or uniformly fused representations, causing task-agnostic propagation and over-compressed fusion that hinder diverse task requirements and modality-specific evidence preservation. To address this, we propose CoMAG, a unified MAG backbone that learns task-adaptive reliable contexts and modality-preserving alignment within them. CoMAG first conducts Reliable Context Learning by estimating edge reliability from multimodal semantic consistency, complementing raw topology with semantic neighbors, and selecting context components through a task-aware gate. It then performs Modality-preserving Hop-token Alignment by maintaining modality-specific multi-hop trajectories, matching modality-hop tokens across modalities, and decoupling shared and private representations. Thus, CoMAG produces graph and modality representations from one forward pass while retaining modality-specific cues. We further analyze stable propagation, over-smoothing mitigation, and modality-collapse control. Experiments on nine OpenMAG datasets compare CoMAG with feature-only, graph-only, multimodal, and unified MAG baselines across graph-level prediction, modality matching, and graph-conditioned generation. Results show that CoMAG achieves the best reported performance, demonstrating that task-adaptive reliable contexts and modality-preserving alignment improve structural prediction, cross-modal matching, and graph-conditioned generation while retaining sparse edge-linear complexity.

Problem

• Multimodal Attributed Graphs
• context-aware
• modality alignment
• task-agnostic propagation
• modality-specific evidence

Innovation

• Context-aware
• Modality-Topology Co-Alignment
• Reliable Context Learning
• Modality-preserving Alignment
• Multimodal Attributed Graphs