Overlapping IPv4, IPv6, and TCP data: exploring errors, test case context and multiple overlaps inside network stacks and NIDSes with PYROLYSE

📅 2025-08-01
📈 Citations: 0
Influential: 0
📄 PDF

career value

194K/year
🤖 AI Summary
Inconsistent reassembly strategies for IP fragmentation and TCP segmentation across network protocol stack implementations cause divergent parsing of overlapping packets between NIDS and monitored hosts, enabling evasion attacks and other security vulnerabilities. Method: We propose PYROLYSE, the first formal testing framework to systematically audit IPv4/IPv6 and TCP reassembly behavior under *n* ≤ 3 overlapping fragments, covering 23 mainstream OSes, NIDS, and embedded protocol stacks. Contribution/Results: Our audit reveals 14–20 distinct behavioral classes and uncovers eight security defects—including CVE-identified vulnerabilities—demonstrating that binary-fragment-based NIDS detection logic fails to generalize to ternary or higher-order overlaps. Crucially, we show that reassembly behavior under multi-fragment overlap cannot be reliably inferred from pairwise fragment tests, underscoring the urgent need for formal verification methods targeting higher-order overlapping scenarios.

Technology Category

Application Category

📝 Abstract
IP fragmentation and TCP segmentation allow for splitting large data packets into smaller ones, e.g., for transmission across network links of limited capacity. These mechanisms permit complete or partial overlaps with different data on the overlapping portions. IPv4, IPv6, and TCP reassembly policies, i.e., the data chunk preferences that depend on the overlap types, differ across protocol implementations. This leads to vulnerabilities, as NIDSes may interpret the packet differently from the monitored host OSes. Some NIDSes, such as Suricata or Snort, can be configured so that their policies are consistent with the monitored OSes. The first contribution of the paper is PYROLYSE, an audit tool that exhaustively tests and describes the reassembly policies of various IP and TCP implementation types. This tool ensures that implementations reassemble overlapping chunk sequences without errors. The second contribution is the analysis of PYROLYSE artifacts. We first show that the reassembly policies are much more diverse than previously thought. Indeed, by testing all the overlap possibilities for n <= 3 test case chunks and different testing scenarios, we observe from 14 to 20 different behaviors out of 23 tested implementations depending on the protocol. Second, we report eight errors impacting one OS, two NIDSes, and two embedded stacks, which can lead to security issues such as NIDS pattern-matching bypass or DoS attacks. A CVE was assigned to a NIDS error. Finally, we show that implemented IP and TCP policies obtained through chunk pair testing are usually inconsistent with the observed triplet reassemblies. Therefore, contrarily to what they currently do, NIDSes or other network traffic analysis tools should not apply n = 2 pair policies when the number of overlapping chunks exceeds two.
Problem

Research questions and friction points this paper is trying to address.

Analyzes IPv4, IPv6, TCP reassembly policy inconsistencies
Identifies vulnerabilities in NIDSes and host OS packet interpretation
Proposes PYROLYSE tool to audit reassembly errors and security risks
Innovation

Methods, ideas, or system contributions that make the work stand out.

PYROLYSE tool audits IP and TCP reassembly policies
Analyzes diverse reassembly behaviors across implementations
Identifies inconsistencies in pair versus triplet reassemblies
L
Lucas Aubard
Inria
J
Johan Mazel
ANSSI
G
Gilles Guette
IMT Atlantique
P
Pierre Chifflier
ANSSI