Who to Trust? Aggregating Client Knowledge in Logit-Based Federated Learning

2025-09-18

AI Summary

To address the challenges of knowledge aggregation across heterogeneous clients, high communication overhead, and poor robustness under non-IID data in logit-based federated learning, this paper proposes a lightweight logit-sharing and adaptive aggregation framework. The method eliminates model parameter transmission by generating comparable logits on a shared proxy dataset. It further introduces three aggregation strategies—simple averaging, uncertainty-weighted fusion, and a learnable meta-aggregation network—to dynamically model client heterogeneity. Extensive experiments on MNIST and CIFAR-10 demonstrate that the proposed approach achieves accuracy comparable to centralized training under non-IID settings, reduces communication cost by an order of magnitude, and significantly outperforms baseline methods. These results validate the framework’s effectiveness, robustness, and generalization capability in practical federated learning scenarios.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Multiagent Systems: Adversarial Agents
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Federated learning (FL) usually shares model weights or gradients, which is costly for large models. Logit-based FL reduces this cost by sharing only logits computed on a public proxy dataset. However, aggregating information from heterogeneous clients is still challenging. This paper studies this problem, introduces and compares three logit aggregation methods: simple averaging, uncertainty-weighted averaging, and a learned meta-aggregator. Evaluated on MNIST and CIFAR-10, these methods reduce communication overhead, improve robustness under non-IID data, and achieve accuracy competitive with centralized training.

Problem

• Aggregating knowledge from heterogeneous clients in federated learning
• Reducing communication costs by sharing logits instead of weights
• Improving robustness and accuracy under non-IID data distributions

Innovation

• Logit-based FL reduces communication cost
• Three aggregation methods compared for robustness
• Achieves competitive accuracy with centralized training