Traffic-MoE: A Sparse Foundation Model for Network Traffic Analysis

2026-01-01

arXiv.org

AI Summary

This work addresses the challenge of deploying large models for real-time, high-throughput cybersecurity analysis, where their substantial computational overhead often hinders practical deployment. To overcome this limitation, the authors propose Traffic-MoE—the first sparse Mixture-of-Experts (MoE) foundation model tailored for network traffic analysis. By leveraging a dynamic token routing mechanism that directs input traffic to a small subset of expert subnetworks, Traffic-MoE integrates pre-trained representation learning with adversarial robustness optimization. This design achieves significant efficiency gains without compromising detection accuracy. Experimental results across three security tasks demonstrate that Traffic-MoE improves detection performance by up to 12.38%, increases throughput by 91.62%, reduces inference latency by 47.81%, and decreases peak GPU memory consumption by 38.72% compared to baseline approaches.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Large Vision Models
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

While pre-trained large models have achieved state-of-the-art performance in network traffic analysis, their prohibitive computational costs hinder deployment in real-time, throughput-sensitive network defense environments. This work bridges the gap between advanced representation learning and practical network protection by introducing Traffic-MoE, a sparse foundation model optimized for high-efficiency real-time inference. By dynamically routing traffic tokens to a small subset of specialized experts, Traffic-MoE effectively decouples model capacity from computational overhead. Extensive evaluations across three security-oriented tasks demonstrate that Traffic-MoE achieves up to a 12.38% improvement in detection performance compared to leading dense competitors. Crucially, it delivers a 91.62% increase in throughput, reduces inference latency by 47.81%, and cuts peak GPU memory consumption by 38.72%. Beyond efficiency, Traffic-MoE exhibits superior robustness against adversarial traffic shaping and maintains high detection efficacy in few-shot scenarios, establishing a new paradigm for scalable and resilient network traffic analysis.

Problem

• network traffic analysis
• computational cost
• real-time inference
• throughput-sensitive
• model deployment

Innovation

• Sparse Foundation Model
• Mixture of Experts
• Real-time Network Traffic Analysis
• Efficient Inference
• Adversarial Robustness