On the Security Implications of PQC in TLS: Handshake Exhaustion and IDS Degradation

📅 2026-07-14
📈 Citations: 0
Influential: 0
📄 PDF
🤖 AI Summary
This study addresses the security-performance trade-offs introduced by integrating post-quantum cryptography (PQC) into TLS 1.3 handshakes, which significantly increases computational overhead, exacerbates DDoS handshake exhaustion attacks, and degrades the efficacy of existing intrusion detection systems (IDS). The authors construct a cloud-based experimental platform supporting PQC-TLS 1.3, combining distributed attack emulation with deep learning–based IDS frameworks such as Exosphere and HyperVision to systematically quantify, for the first time, the impact of PQC on attack persistence and IDS performance. Results reveal that PQC extends periods of high server CPU utilization by up to 88× and reduces mainstream IDS recall to approximately 50%, with AU-ROC scores degrading to 0.49—near random classification levels. The work also releases an open-source, timestamped PQC-DDoS hybrid traffic dataset with fine-grained resource monitoring and a fully reproducible testbed.
📝 Abstract
Post-Quantum Cryptography (PQC) is increasingly being integrated into TLS 1.3 to enhance resilience against quantum-enabled attacks. However, the additional computational and communication overhead introduced by PQC primitives during the handshake phase may also amplify the impact of TLS handshake exhaustion attacks, leading to more severe Distributed Denial-of-Service (DDoS) threats. In this study, we establish an empirical testbed consisting of one PQC-enabled TLS server and ten attacking nodes, generating over 16.5 GB of mixed traffic data that includes both legitimate browsing behavior and high-intensity handshake exhaustion attacks. Experimental results show that PQC-TLS can prolong periods of sustained high CPU utilization on the server by up to 88 times, significantly amplifying the effectiveness of such attacks. Furthermore, we evaluate state-of-the-art deep learning-based Intrusion Detection Systems (IDS) and observe a substantial decline in attack detection performance under PQC traffic conditions. In particular, exosphere achieves only around 50% recall, while HyperVision's AU-ROC degrades to near-random levels (0.49), revealing critical detection blind spots in existing IDS when operating in PQC environments. The main contributions of this work are threefold: (1) we systematically quantify and analyze the root causes of IDS detection blind spots in PQC settings; (2) we publicly release a comprehensive PQC-DDoS hybrid traffic dataset, including precise attack timestamps and server-side resource monitoring data; and (3) we open-source all experimental code and AWS deployment scripts, enabling a fully reproducible cloud-based testing environment. These resources aim to support both academia and industry in developing next-generation PQC-aware intrusion detection systems.
Problem

Research questions and friction points this paper is trying to address.

Post-Quantum Cryptography
TLS handshake exhaustion
DDoS
Intrusion Detection System
Security Implications
Innovation

Methods, ideas, or system contributions that make the work stand out.

Post-Quantum Cryptography
TLS Handshake Exhaustion
Intrusion Detection System
DDoS
Reproducible Dataset
🔎 Similar Papers
No similar papers found.
L
Lin-Fa Lee
Department of Institute of Artificial Intelligence Innovation, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
Y
Yi-Yu Chang
Department of Institute of Artificial Intelligence Innovation, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
Chia-Mu Yu
Chia-Mu Yu
National Yang Ming Chiao Tung University
AI SecurityData PrivacyData AnonymizationCryptography
Kuo-Hui Yeh
Kuo-Hui Yeh
National Yang Ming Chiao Tung University
SecurityPrivacy