Beyond the Aggregation Dilemma: Prior-Retaining Decoupled Learning for Multimodal Graphs

2026-05-23

AI Summary

In the context of large foundation models providing strong priors, conventional multimodal graph learning underperforms even simple MLPs due to topological noise induced by forced aggregation and resultant gradient starvation. This work is the first to identify and systematically analyze this “aggregation dilemma,” proposing SUPRA, a decoupled dual-path architecture: one path preserves modality-specific priors via a topology-agnostic MLP, while the other captures structural synergy through a lightweight shared GNN, complemented by deep supervision to alleviate gradient starvation. The method achieves state-of-the-art performance across multiple benchmarks, reducing peak GPU memory consumption by 3.5× and accelerating training by up to 4.4× compared to multimodal graph Transformers.

Abstract

Multimodal Attributed Graph Learning (MAGL) integrates intrinsic node attributes with structural topology via graph aggregation. However, as pretrained encoders evolve into Large Foundation Models (LFMs), the landscape of MAGL fundamentally shifts: under high-confidence LFM priors, mandatory aggregation introduces topological noise that overwhelms discriminative signals, triggering a counter-intuitive performance inversion where sophisticated MAGL architectures underperform simple topology-agnostic MLPs. Through systematic empirical and theoretical analysis, we identify that this inversion stems from a fundamental aggregation dilemma characterized by two concurrent pathologies: (1) Representational Pathology (SNR Degradation) - mandatory aggregation dilutes robust intrinsic features with topological noise, causing the noise penalty to outweigh its collaborative benefit; and (2) Optimization Pathology (Gradient Starvation) - topological aggregation attenuates gradient flow, while a shared task loss causes dominant modalities to prematurely suppress weaker ones. To resolve this dilemma, we propose SUPRA (Shared-Unique Prior-Retaining Architecture), a decoupled dual-pathway paradigm. SUPRA processes modality-specific features through topology-agnostic MLPs while capturing structural synergy via a lightweight shared GNN, with auxiliary deep supervision counteracting gradient starvation. Extensive evaluations demonstrate that SUPRA achieves state-of-the-art performance while requiring 3.5x lower peak GPU memory and up to 4.4x faster training time than Multimodal Graph Transformers.

Problem

• Multimodal Graph Learning
• Aggregation Dilemma
• Topological Noise
• Gradient Starvation
• Large Foundation Models

Innovation

• Decoupled Learning
• Prior-Retaining
• Multimodal Graph Learning
• Aggregation Dilemma
• Gradient Starvation