Federated Learning for Feature Generalization with Convex Constraints

2026-06-12

AI Summary

This work addresses the challenge of degraded model generalization and distorted transferable features in federated learning caused by client data heterogeneity. To mitigate these issues, the authors propose FedCONST, a novel approach that integrates a parameter magnitude-aware adaptive update mechanism with linear convex constraints to effectively align local and global optimization objectives during aggregation. By adaptively modulating update magnitudes and incorporating gradient signal-to-noise ratio (GSNR) analysis, FedCONST significantly enhances training stability, alleviates overfitting, and achieves state-of-the-art generalization performance across diverse heterogeneous federated settings. The method further improves feature transferability and model robustness without compromising convergence efficiency.

Abstract

Federated learning (FL) often struggles with generalization due to heterogeneous client data. Local models are prone to overfitting their local data distributions, and even transferable features can be distorted during aggregation. To address these challenges, we propose FedCONST, an approach that adaptively modulates update magnitudes based on the parameter strength of the global model. This prevents over-emphasizing well-learned parameters while reinforcing underdeveloped ones. Specifically, FedCONST employs linear convex constraints to ensure training stability and preserve locally learned generalization capabilities during aggregation. A Gradient Signal to Noise Ratio (GSNR) analysis further validates the effectiveness of FedCONST in enhancing feature transferability and robustness. As a result, FedCONST effectively aligns local and global objectives, mitigating overfitting and promoting stronger generalization across diverse FL environments, achieving state-of-the-art performance.

Problem

• Federated Learning
• Feature Generalization
• Data Heterogeneity
• Model Aggregation
• Overfitting

Innovation

• Federated Learning
• Feature Generalization
• Convex Constraints
• Gradient Signal-to-Noise Ratio
• Adaptive Update Modulation