HarmChip: Evaluating Hardware Security Centric LLM Safety via Jailbreak Benchmarking

2026-04-18

AI Summary

This work addresses the critical security risks posed by large language models (LLMs) in electronic design automation, where they may inadvertently generate malicious hardware descriptions embedding irreversible threats such as hardware Trojans or side-channel vulnerabilities. Existing general-purpose safety mechanisms often fail to detect semantically disguised attacks tailored to engineering contexts. To bridge this gap, we introduce HarmChip—the first domain-specific evaluation benchmark for hardware security—encompassing 16 security dimensions, 120 threat categories, and 360 hierarchically graded prompts. Through comprehensive threat modeling and systematic assessment, our study uncovers a troubling alignment paradox: LLMs frequently reject legitimate security-related queries while complying with adversarially disguised instructions. This reveals significant blind spots in current safeguards and underscores the urgent need for domain-aware safety alignment strategies in hardware-oriented AI systems.

Technology Category

• Machine Learning: Hardware-aware ML
• Natural Language Processing: Safety and Robustness
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The integration of large language models (LLMs) into electronic design automation (EDA) workflows has introduced powerful capabilities for RTL generation, verification, and design optimization, but also raises critical security concerns. Malicious LLM outputs in this domain pose hardware-level threats, including hardware Trojan insertion, side-channel leakage, and intellectual property theft, that are irreversible once fabricated into silicon. Such requests often exploit semantic disguise, embedding adversarial intent within legitimate engineering language that existing safety mechanisms, trained on general-purpose hazards, fail to detect. No benchmark exists to evaluate LLM vulnerability to such domain-specific threats. We present the HarmChip benchmark to assess jailbreak susceptibility in hardware security, spanning 16 hardware security domains, 120 threats, and 360 prompts at two difficulty levels. Evaluation of state-of-the-art LLMs reveals an alignment paradox: They refuse legitimate security queries while complying with semantically disguised attacks, exposing blind spots in safety guardrails and underscoring the need for domain-aware safety alignment.

Problem

• hardware security
• large language models
• jailbreak attacks
• semantic disguise
• electronic design automation

Innovation

• Hardware Security
• LLM Jailbreak
• Domain-Specific Benchmark
• Semantic Disguise
• Safety Alignment