From Coordinate Matching to Structural Alignment: Rethinking Prototype Alignment in Heterogeneous Federated Learning

2026-05-07

AI Summary

Existing prototype alignment methods in heterogeneous federated learning enforce clients with diverse architectures to align within a unified feature subspace, thereby constraining model expressiveness. This work proposes FedSAF, a novel structural alignment paradigm that shifts the alignment objective from coordinate-wise matching to preserving the consistency of inter-class relational structures. By decoupling semantic structure alignment from shared feature bases, FedSAF models class relationships through prototypes and integrates them into a distributed optimization framework. Extensive experiments demonstrate that FedSAF significantly outperforms current heterogeneous federated learning approaches across multiple benchmarks, achieving accuracy improvements of up to 3.52%.

Abstract

Heterogeneous federated learning (HtFL) aims to enable collaboration among clients that differ in both data distributions and model architectures. Prototype-based methods, which communicate class-level feature centers (prototypes) instead of full model parameters, have recently shown strong potential for HtFL. Existing prototype-based HtFL methods typically reuse the MSE-based or cosine-based alignment mechanism developed for homogeneous FL when aligning client-specific representations with global prototypes. These approaches are essentially coordinate alignment, where representations of clients are forced to match the global prototypes in the embedding space in an element-wise manner. Such alignment implicitly assumes that all clients should map their representations into the feature subspace defined by the global prototypes. This assumption is reasonable in homogeneous FL, where all clients share the same feature extractor. However, it becomes problematic in HtFL, since heterogeneous feature extractors naturally induce client-specific feature subspaces, and forcing all clients to optimize within a single global subspace unnecessarily suppresses their learning capacity. We observe that coordinate alignment implicitly couples two distinct objectives: aligning inter-class semantic structure, which is directly beneficial for classification, and enforcing a shared feature basis, which is unnecessary and even harmful under model heterogeneity. Building on this insight, we design FedSAF, which shifts the alignment objective from absolute coordinates to inter-class relational structure. We demonstrate that structural alignment consistently outperforms coordinate alignment in heterogeneous settings. Experiments on multiple benchmarks show that our structural alignment outperforms state-of-the-art prototype-based HtFL methods by up to 3.52\%.

Problem

• heterogeneous federated learning
• prototype alignment
• coordinate matching
• structural alignment
• feature subspace

Innovation

• structural alignment
• heterogeneous federated learning
• prototype-based methods
• feature subspace
• FedSAF