From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneity

2026-04-09

AI Summary

Federated continual learning in dynamic and heterogeneous environments is highly susceptible to class imbalance, which often leads to representation collapse and degraded performance on minority classes. To address this challenge, this work proposes FEAT, the first approach to integrate a geometry-aware correction mechanism into federated rehearsal-based learning. FEAT constructs a shared, discriminative geometric reference structure across clients using equiangular tight frame (ETF) prototypes, aligns feature spaces via structured knowledge distillation, and removes task-irrelevant directional components through an energy-based geometric correction. This methodology effectively mitigates representation drift and majority-class bias, significantly enhancing model robustness and sensitivity to minority classes in federated continual learning settings.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Representation Learning for Vision
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Exemplar replay has become an effective strategy for mitigating catastrophic forgetting in federated continual learning (FCL) by retaining representative samples from past tasks. Existing studies focus on designing sample-importance estimation mechanisms to identify information-rich samples. However, they typically overlook strategies for effectively utilizing the selected exemplars, which limits their performance under continual dynamic heterogeneity across clients and tasks. To address this issue, this paper proposes a Federated gEometry-Aware correcTion method, termed FEAT, which alleviates imbalance-induced representation collapse that drags rare-class features toward frequent classes across clients. Specifically, it consists of two key modules: 1) the Geometric Structure Alignment module performs structural knowledge distillation by aligning the pairwise angular similarities between feature representations and their corresponding Equiangular Tight Frame prototypes, which are fixed and shared across clients to serve as a class-discriminative reference structure. This encourages geometric consistency across tasks and helps mitigate representation drift; 2) the Energy-based Geometric Correction module removes task-irrelevant directional components from feature embeddings, which reduces prediction bias toward majority classes. This improves sensitivity to minority classes and enhances the model's robustness under class-imbalanced distributions.

Problem

• federated continual learning
• exemplar replay
• dynamic heterogeneity
• class imbalance
• representation collapse

Innovation

• Federated Continual Learning
• Exemplar Replay
• Geometry-Aware Correction
• Equiangular Tight Frame
• Class Imbalance