FlowletFormer: Network Behavioral Semantic Aware Pre-training Model for Traffic Classification

2025-08-27

AI Summary

Existing network traffic pretraining models struggle to jointly model packet-level structure, flow-level behavioral patterns, hierarchical protocol semantics, and cross-packet/cross-flow contextual relationships. To address this, we propose FlowletFormer—the first BERT-style pretraining model specifically designed for network traffic. It introduces three key innovations: (1) behavior-semantic-aware flow segmentation, (2) protocol-stack-aligned embedding layers, and (3) field-specific and context-aware joint pretraining objectives. These components systematically integrate domain knowledge of networking protocols, significantly enhancing semantic understanding—particularly for stateful protocols such as TCP. Extensive experiments demonstrate that FlowletFormer consistently outperforms prior methods across representation quality, classification accuracy, and few-shot generalization capability. Moreover, it exhibits superior robustness and practicality in complex, real-world network scenarios.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Large Vision Models

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Network traffic classification using pre-training models has shown promising results, but existing methods struggle to capture packet structural characteristics, flow-level behaviors, hierarchical protocol semantics, and inter-packet contextual relationships. To address these challenges, we propose FlowletFormer, a BERT-based pre-training model specifically designed for network traffic analysis. FlowletFormer introduces a Coherent Behavior-Aware Traffic Representation Model for segmenting traffic into semantically meaningful units, a Protocol Stack Alignment-Based Embedding Layer to capture multilayer protocol semantics, and Field-Specific and Context-Aware Pretraining Tasks to enhance both inter-packet and inter-flow learning. Experimental results demonstrate that FlowletFormer significantly outperforms existing methods in the effectiveness of traffic representation, classification accuracy, and few-shot learning capability. Moreover, by effectively integrating domain-specific network knowledge, FlowletFormer shows better comprehension of the principles of network transmission (e.g., stateful connections of TCP), providing a more robust and trustworthy framework for traffic analysis.

Problem

• Captures packet structural characteristics and flow-level behaviors
• Integrates hierarchical protocol semantics and inter-packet relationships
• Enhances traffic representation accuracy and few-shot learning capability

Innovation

• Coherent Behavior-Aware Traffic Representation Model
• Protocol Stack Alignment-Based Embedding Layer
• Field-Specific and Context-Aware Pretraining Tasks