Towards Performance-Enhanced Model-Contrastive Federated Learning using Historical Information in Heterogeneous Scenarios

2026-02-12

AI Summary

This work addresses the challenges of poor convergence and performance instability in federated learning under data heterogeneity and imbalanced client participation. To mitigate these issues, the authors propose the PMFL framework, which introduces a novel integration of historical local models into contrastive learning to enhance update consistency. The framework adaptively adjusts aggregation weights based on each client’s cumulative participation count and further incorporates historical global models to improve training stability. By synergistically combining model contrast, adaptive weighting, and historical model fusion, PMFL achieves significantly superior performance across diverse heterogeneous settings, effectively accelerating convergence and enhancing model generalization compared to existing approaches.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Multiagent Systems: Multiagent Learning
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Federated Learning (FL) enables multiple nodes to collaboratively train a model without sharing raw data. However, FL systems are usually deployed in heterogeneous scenarios, where nodes differ in both data distributions and participation frequencies, which undermines the FL performance. To tackle the above issue, this paper proposes PMFL, a performance-enhanced model-contrastive federated learning framework using historical training information. Specifically, on the node side, we design a novel model-contrastive term into the node optimization objective by incorporating historical local models to capture stable contrastive points, thereby improving the consistency of model updates in heterogeneous data distributions. On the server side, we utilize the cumulative participation count of each node to adaptively adjust its aggregation weight, thereby correcting the bias in the global objective caused by different node participation frequencies. Furthermore, the updated global model incorporates historical global models to reduce its fluctuations in performance between adjacent rounds. Extensive experiments demonstrate that PMFL achieves superior performance compared with existing FL methods in heterogeneous scenarios.

Problem

• Federated Learning
• Heterogeneous Scenarios
• Data Distribution
• Participation Frequency
• Model Performance

Innovation

• model-contrastive learning
• federated learning
• heterogeneous scenarios
• historical information
• adaptive aggregation