Sheaf-Based Federated Representation Learning

2026-08-08

AI Summary

This work addresses the challenge of representation alignment in heterogeneous federated learning, where discrepancies in data distributions, sensing modalities, model architectures, and latent space dimensions hinder effective collaboration. To overcome this, the authors propose a sheaf-based federated representation learning framework that dispenses with the assumption of a globally shared latent space. Instead, it introduces learnable sheaf restriction maps and a sheaf Laplacian-induced quadratic gluing regularizer, enabling geometric alignment of neighboring nodes via orthogonal transformations and isometric embeddings under manifold constraints. The resulting decentralized Sheaf-FRL algorithm enjoys theoretical convergence guarantees and maintains communication efficiency even with minimal shared samples. Empirical results demonstrate its superior performance over baseline methods in semantic communication-assisted collaborative classification, achieving higher local and post-communication accuracy while exhibiting greater robustness to latent space compression.

Abstract

Heterogeneous federated systems require agents to learn and exchange informative representations despite differences in data distributions, sensing modalities, model architectures, latent dimensionalities, and local learning objectives. To address this challenge, we propose Sheaf-based Federated Representation Learning (SFRL), a general framework that jointly optimizes local objectives with a manifold-constrained geometric alignment regularizer based on learnable sheaf restriction maps. Unlike most existing approaches, SFRL does not assume a shared global latent space. Instead, global consistency emerges from the alignment of neighboring latent representations through orthogonal transformations and isometric embeddings. This alignment is enforced by a quadratic gluing regularizer induced by the sheaf Laplacian, whose learnable restriction maps adapt the geometry to the observed data. The penalty is evaluated on a small set of shared pilot samples, ensuring scalability and communication efficiency. We develop a decentralized algorithm for solving SFRL, termed Sheaf-FRL, which alternates between gradient updates of the local models and closed-form Procrustes updates of the edge-wise restriction maps. We further establish convergence of Sheaf-FRL to first-order stationary points in both deterministic and stochastic settings. As an application, we consider a cooperative classification task in the context of semantic communication, under model and data heterogeneity. Our results show that Sheaf-FRL outperforms baseline approaches in terms of local and post-communication classification accuracy across different levels of local distribution shift and exhibits greater robustness to latent-space dimensionality compression.

Problem

• heterogeneous federated learning
• representation learning
• data heterogeneity
• model heterogeneity
• latent space alignment

Innovation

• Sheaf-based learning
• Federated representation learning
• Geometric alignment
• Manifold-constrained optimization
• Heterogeneous federated learning