Reducing Communication Overhead in Federated Learning for Network Anomaly Detection with Adaptive Client Selection

2025-03-19

AI Summary

To address the challenge in federated learning for network intrusion detection—where client heterogeneity incurs excessive communication overhead and compromises the trade-off between accuracy and efficiency—this paper proposes the first integrated framework synergizing dynamic batch-size optimization, adaptive client selection, and asynchronous model updates. Guided by Mann–Whitney U statistical testing for strategy design and enhanced via GPU operator profiling and memory-transfer analysis, the framework is rigorously validated across heterogeneous domains (UNSW-NB15 and ROAD datasets). Experimental results demonstrate a 97.6% reduction in communication time (from 700.0 s to 16.8 s), while maintaining a stable detection accuracy of 95.10%—statistically indistinguishable from the centralized baseline (95.12%, *p* < 0.05). To our knowledge, this is the first work achieving Pareto-optimal balance between communication cost and model performance in federated network intrusion detection.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Distributed Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Communication overhead in federated learning (FL) poses a significant challenge for network anomaly detection systems, where diverse client configurations and network conditions impact efficiency and detection accuracy. Existing approaches attempt optimization individually but struggle to balance reduced overhead with performance. This paper presents an adaptive FL framework combining batch size optimization, client selection, and asynchronous updates for efficient anomaly detection. Using UNSW-NB15 for general network traffic and ROAD for automotive networks, our framework reduces communication overhead by 97.6% (700.0s to 16.8s) while maintaining comparable accuracy (95.10% vs. 95.12%). The Mann-Whitney U test confirms significant improvements (p<0.05). Profiling analysis reveals efficiency gains via reduced GPU operations and memory transfers, ensuring robust detection across varying client conditions.

Problem

• Reduces communication overhead in federated learning for anomaly detection.
• Balances efficiency and accuracy in diverse client configurations.
• Proposes adaptive client selection and asynchronous updates for optimization.

Innovation

• Adaptive client selection reduces communication overhead.
• Combines batch size optimization and asynchronous updates.
• Maintains high accuracy with reduced GPU operations.