ANTMAN: An Efficient and Interpretable RTL-Level Run-Time Detection Framework for Stealthy Branch Predictor Attacks on BOOM

2026-08-10

AI Summary

This work addresses the lack of effective runtime detection mechanisms against covert branch predictor side-channel attacks on RISC-V platforms. Existing performance counter–based approaches struggle to balance accuracy, speed, and granularity, and fail to defend against attacks that bypass caches by directly exploiting branch history table states. To overcome these limitations, the paper proposes a security-first, interpretable, and non-intrusive RTL-level runtime detection framework. Implemented on the BOOM processor, the approach leverages offline mining of branch behavior association rules and integrates lightweight hardware monitors to enable real-time threat interception. This method achieves the first RTL-level, non-intrusive detection on RISC-V BOOM, supports both Next-Line Predictor and TAGE configurations, terminates threats before attack execution, incurs zero false positives, and generalizes effectively to emerging variants of such attacks, significantly outperforming existing solutions.

Abstract

Runtime detection of microarchitectural side channel attacks remains significantly underexplored in RISCV compared with x86 and ARM ISAs, posing a serious threat to critical applications. State-of-the-art branch predictor attacks bypass traditional data and instruction caches by directly exploiting the state of internal history tables, making them inherently stealthy. Recent research has explored offline detection of microarchitectural attacks on RISC-V; however, efficient runtime detection of microarchitectural attacks on RISC-V hardware remains significantly unaddressed. State-of-the-art hardware-based runtime detection solutions leverage hardware performance counters (HPCs) but suffer from a restricted set of counter registers and tradeoffs between detection accuracy, detection speed, and sampling granularity, making them impractical for stealthy attacks. Moreover, sampling HPCs after distinct intervals leaves intermediate relationships between different microarchitectural blocks unobserved. Additionally, proprietary x86 and ARM ISAs constrain researchers from modifying processor microarchitectural designs. To address these limitations, we propose the first secure-by-design, highly interpretable, non-intrusive, RTL-level runtime detection solution for stealthy branch predictor attacks on BOOM RISC-V, evaluated under both simplified Next-Line Predictor (NLP) and complex TAGE predictor configurations. The attack detection relies on association rules extracted offline and embedded in hardware as a non-intrusive rule monitor that enables runtime detection. The proposed approach achieves excellent detection speed, terminates execution before secret disclosure, and produces zero false positives while remaining flexible for detecting previously unseen variants within the same family of branch predictor attacks.

Problem

• branch predictor attacks
• microarchitectural side channel
• runtime detection
• RISC-V
• stealthy attacks

Innovation

• RTL-level detection
• branch predictor attacks
• RISC-V BOOM
• non-intrusive monitoring
• association rule mining