FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning

2026-06-30

AI Summary

This work addresses the challenge of learning from multimodal graph data in federated settings, where privacy constraints prevent centralized data sharing and existing methods struggle to trace the joint decision-making process involving modality evidence, node semantics, and topological context. To this end, the authors propose FedLAB, a novel framework that introduces, for the first time, a typed hierarchical semantic codebook equipped with a native semantic-traceable interface. FedLAB enables explicit modeling of the joint reasoning pathways among modalities, semantics, and topology through federated semantic centroid pretraining, all while preserving data locality. Extensive experiments demonstrate that FedLAB consistently outperforms state-of-the-art methods by up to 7.53% on average across ten benchmark datasets and six downstream tasks, achieving a unified balance between high predictive accuracy and semantic traceability.

Abstract

Multimodal graph foundation models aim to learn reusable knowledge from graphs enriched with text, images, attributes, and relational topology, thereby supporting diverse graph-centric and modality-centric tasks. In practice, however, such multimodal graphs are often distributed across decentralized clients, where raw contents and local structures cannot be centrally shared due to privacy constraints. This motivates federated multimodal graph foundation learning, which requires not only transferable representation learning but also intrinsic semantic traceability under strict data isolation. Existing methods usually exchange or store knowledge through parameters, prototypes, embeddings, or compact codebooks, which support optimization and transfer but do not explicitly expose how modality evidence, node semantics, and topology context jointly support predictions. To bridge this gap, we propose FedLAB, a traceable semantic codebook framework that organizes multimodal graph knowledge into typed hierarchical codebooks for modality evidence, node semantics, and topology context. FedLAB further refines these trace units through federated semantic barycenter pre-training while keeping raw multimodal contents and graph structures local. Extensive experiments on 10 benchmarks and 6 downstream tasks show that FedLAB improves over state-of-the-art baselines by up to 7.53\%, while preserving a native semantic trace interface.

Problem

• federated learning
• multimodal graph
• semantic traceability
• data isolation
• foundation model

Innovation

• federated learning
• multimodal graph
• semantic codebook
• traceability
• foundation model