Resource-Constrained Federated Continual Learning: What Does Matter?

2025-01-15

AI Summary

To address the limited practicality of Federated Continual Learning (FCL) on resource-constrained edge devices—characterized by scarce storage, limited computational capacity, and low annotation rates—this work introduces the first large-scale FCL benchmark explicitly designed for resource constraints, covering six diverse datasets under both class-incremental and domain-incremental settings. Through systematic experiments consuming over 1,000 GPU hours, we evaluate the performance degradation of mainstream FCL approaches—including replay-based, regularization-based, and parameter-isolation methods—across multiple resource dimensions. Results reveal significant performance collapse under resource constraints, exposing their inherent dependence on abundant resources. Consequently, we propose a lightweight-deployment-oriented FCL evaluation paradigm and establish joint resource-performance optimization as a core research direction. This work provides critical insights, a reproducible benchmark, and methodological foundations to advance practical FCL deployment.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Computer Vision: Learning & Optimization for CV

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Federated Continual Learning (FCL) aims to enable sequentially privacy-preserving model training on streams of incoming data that vary in edge devices by preserving previous knowledge while adapting to new data. Current FCL literature focuses on restricted data privacy and access to previously seen data while imposing no constraints on the training overhead. This is unreasonable for FCL applications in real-world scenarios, where edge devices are primarily constrained by resources such as storage, computational budget, and label rate. We revisit this problem with a large-scale benchmark and analyze the performance of state-of-the-art FCL approaches under different resource-constrained settings. Various typical FCL techniques and six datasets in two incremental learning scenarios (Class-IL and Domain-IL) are involved in our experiments. Through extensive experiments amounting to a total of over 1,000+ GPU hours, we find that, under limited resource-constrained settings, existing FCL approaches, with no exception, fail to achieve the expected performance. Our conclusions are consistent in the sensitivity analysis. This suggests that most existing FCL methods are particularly too resource-dependent for real-world deployment. Moreover, we study the performance of typical FCL techniques with resource constraints and shed light on future research directions in FCL.

Problem

• Federated Continual Learning
• Resource Efficiency
• Performance Optimization

Innovation

• Federated Continual Learning
• Resource Efficiency
• Performance Evaluation