Fog Intelligence for Network Anomaly Detection

2020-03-01

IEEE Network

AI Summary

Detecting anomalous behaviors in large-scale mobile communication networks is challenging due to the high dimensionality and distributed nature of monitoring data. Method: This paper proposes a fog-intelligence architecture that integrates lightweight edge inference with cloud-based collaborative learning. It innovatively unifies federated learning, distributed machine learning, and edge computing to jointly address scalability, privacy preservation, and real-time responsiveness—overcoming deployment bottlenecks of conventional centralized models in wireless networks. Contribution/Results: Through lightweight model design and cross-layer cooperative optimization, the architecture significantly improves both timeliness and accuracy of anomaly detection. It achieves millisecond-level response and high-precision identification across networks with up to ten million endpoints, enabling real-time, secure, and scalable intelligent network management.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Multiagent Systems: Agent Communication

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Large-scale security measurements
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Anomalies are common in network system monitoring. When manifested as network threats to be mitigated, service outages to be prevented, and security risks to be ameliorated, detecting such anomalous network behaviors becomes of great importance. However, the growing scale and complexity of the mobile communication networks, as well as the ever-increasing amount and dimensionality of network surveillance data, make it extremely difficult to monitor a mobile network and discover abnormal network behaviors. Recent advances in machine learning allow obtaining near-optimal solutions to complicated decision making problems with many sources of uncertainty that cannot be accurately characterized by traditional mathematical models. However, most machine learning algorithms are centralized, which renders them inapplicable to large-scale distributed wireless networks with tens of millions of mobile devices. In this article, we present fog intelligence, a distributed machine learning architecture that enables intelligent wireless network management. It preserves the advantage of both edge processing and centralized cloud computing. Furthermore, the proposed architecture is scalable, privacy-preserving, and well suited for intelligent management of a distributed wireless network.

Problem

• Detecting network anomalies in large-scale distributed systems
• Overcoming limitations of centralized machine learning in wireless networks
• Enabling scalable privacy-preserving intelligent network management

Innovation

• Distributed machine learning for network management
• Combines edge processing and cloud computing
• Scalable and privacy-preserving anomaly detection