Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learning

2026-08-12

AI Summary

This work addresses the performance degradation in federated learning under data heterogeneity, where conventional aggregation based on parameter-space similarity often fails. The authors propose LIGHTYEAR, a novel framework that introduces Neural Tangent Kernels (NTK) into federated learning to measure consistency between model updates and each client’s target-domain predictive behavior in function space. By leveraging a peer-to-peer topology and enabling clients to select beneficial updates using their private validation data, LIGHTYEAR replaces parameter-space similarity with function-space alignment, thereby enabling personalized, target-aware aggregation. Experimental results across five datasets demonstrate that LIGHTYEAR significantly outperforms nine state-of-the-art baselines, achieving substantial gains in both model performance and robustness in heterogeneous settings.

Abstract

Federated learning (FL) enables collaborative model training across distributed clients while keeping data local. A central challenge is determining which client updates are beneficial for aggregation with respect to each client's target domain. Existing methods typically address this problem in parameter space by comparing model parameters or gradients. However, parameter-space similarity can be a poor proxy for predictive behavior, especially under heterogeneous, non-IID data. Consequently, updates that are misaligned with a client's target domain, including those caused by heterogeneous data or malfunctioning clients, may degrade local model performance. We propose Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space. LIGHTYEAR uses an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client. By relating model parameters to local predictive responses, the Neural Tangent Kernel (NTK) provides a more expressive criterion for update selection than parameter-space similarity alone. Because function-space information is not available before aggregation in conventional centralized FL, LIGHTYEAR uses a peer-to-peer (P2P) topology in which clients exchange updates directly and evaluate incoming models on private validation data. Each client selects only updates that are beneficial for its own target domain and aggregates them using a regularized rule that improves stability under heterogeneity. Across five datasets and nine baseline methods, LIGHTYEAR consistently outperforms centralized FL baselines and existing P2P approaches.

Problem

• Federated Learning
• Non-IID Data
• Personalized Aggregation
• Function Space
• Model Update Selection

Innovation

• Neural Tangent Kernel
• function space
• personalized aggregation
• peer-to-peer federated learning
• heterogeneous data