🤖 AI Summary
This study addresses the challenge of mapping cybersecurity course keywords to the Cybersecurity Body of Knowledge (CyBOK) due to ambiguous, overly broad terminology and incomplete alignment. To overcome this, the authors propose an interpretable human-in-the-loop retrieval framework that employs multi-level semantic strategies—including query normalization, manual term expansion, concept weight enhancement, enriched topic descriptions, and domain-sensitive ranking—to generate Top-k candidate CyBOK entries for expert review, thereby avoiding reliance on strict exact matching. The work introduces the ECA-5 evaluation metric, which assesses mapping utility based on expert judgment. Experimental results demonstrate that the proposed approach achieves a 98.00% ECA-5 accuracy on the validation set, significantly outperforming purely structural matching methods and effectively supporting experts in performing efficient and reliable knowledge alignment.
📝 Abstract
This paper presents CyBOKClaw, an interpretable human-in-the-loop retrieval framework for mapping cybersecurity keywords or phrases (KWoPs) to the Cyber Security Body of Knowledge (CyBOK). Rather than treating the task as strict exact classification, the framework is designed as a top-k candidate generator for expert review. It combines query normalization, curated term expansion, concept-level boosts, topic-description enrichment, and domain-sensitive ranking rules. Because educational KWoPs are often broad, ambiguous, and only approximately aligned with CyBOK terminology, strict exact matching provides only a partial account of practical utility. We therefore evaluate the framework using both structural retrieval metrics and an expert-guided top-5 usefulness metric, ECA-5 (Exact or Closest Acceptable Match at top-5), which records whether the returned candidates contain at least one mapping that an expert would judge exact or accept as the nearest practical CyBOK placement. On the development dataset, CyBOKClaw achieves 64.73% EXA-5 (Exact Match at top-5), 84.18% structural semantic alignment, and 91.88% ECA-5; on the validation dataset, it achieves 81.19% EXA-5, 93.32% structural semantic alignment, and 98.00% ECA-5. These results show that expert-guided top-k usefulness provides a more faithful account of practical CyBOK mapping utility than exact structural matching alone, and that CyBOKClaw is effective as a CyBOK-specific expert-support retrieval system.