Closing the Generalization Gap in Parameter-efficient Federated Edge Learning

2025-11-28

AI Summary

To address the degradation of model generalization in Federated Edge Learning (FEEL) caused by data heterogeneity and resource constraints, this paper proposes a generalization-aware, parameter-efficient joint optimization framework. Methodologically, it innovatively leverages an information-theoretic generalization bound to guide convergence analysis and formulates, for the first time, a generalization-aware optimization problem targeting minimization of the upper bound on the average squared gradient norm—thereby enabling joint optimization of model pruning, client selection, and communication-computation resource allocation. A mixed-integer non-convex model is developed and solved via an alternating optimization algorithm under energy and latency constraints. Experimental results across multiple benchmarks demonstrate that the proposed method significantly reduces the generalization gap, accelerates model convergence, and improves resource utilization efficiency—outperforming state-of-the-art approaches.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Federated edge learning (FEEL) provides a promising foundation for edge artificial intelligence (AI) by enabling collaborative model training while preserving data privacy. However, limited and heterogeneous local datasets, as well as resource-constrained deployment, severely degrade both model generalization and resource utilization, leading to a compromised learning performance. Therefore, we propose a parameter-efficient FEEL framework that jointly leverages model pruning and client selection to tackle such challenges. First, we derive an information-theoretic generalization statement that characterizes the discrepancy between training and testing function losses and embed it into the convergence analysis. It reveals that a larger local generalization statement can undermine the global convergence. Then, we formulate a generalization-aware average squared gradient norm bound minimization problem, by jointly optimizing the pruning ratios, client selection, and communication-computation resources under energy and delay constraints. Despite its non-convexity, the resulting mixed-integer problem is efficiently solved via an alternating optimization algorithm. Extensive experiments demonstrate that the proposed design achieves superior learning performance than state-of-the-art baselines, validating the effectiveness of coupling generalization-aware analysis with system-level optimization for efficient FEEL.

Problem

• Addressing model generalization degradation in federated edge learning
• Optimizing resource utilization under energy and delay constraints
• Jointly solving model pruning and client selection challenges

Innovation

• Jointly leverages model pruning and client selection
• Formulates generalization-aware gradient norm minimization
• Solves optimization via alternating algorithm under constraints