Multi-Class vs. Multi-Label BERT for CVE-to-CWE Mapping: How Taxonomy Structure Shapes the Errors

2026-07-08

AI Summary

This study addresses the ongoing challenge of automatically mapping Common Vulnerabilities and Exposures (CVEs) to Common Weakness Enumerations (CWEs), particularly the lack of systematic evaluation in choosing between multi-class and multi-label modeling strategies. Framing the task as text classification, the authors compare these two approaches using three transformer-based encoders—BERT Base, SecureBERT, and CySecBERT—across CWE label spaces of varying granularity. They further introduce a hierarchical relaxed evaluation metric to more accurately assess model performance within the structured hierarchy of CWEs. Experimental results show that the multi-class approach generally achieves higher macro F1 scores, though this advantage diminishes as the label space narrows. CySecBERT performs best under the multi-label setting. Notably, the hierarchical relaxed metric elevates macro F1 from approximately 81% to 90%, indicating that most misclassifications stem from ambiguities in the CWE taxonomy rather than model limitations.

Abstract

Assigning Common Weakness Enumeration (CWE) categories to Common Vulnerabilities and Exposures (CVE) records remains an important but largely manual step in vulnerability analysis. We study this task as a text classification problem and compare two modelling choices: a \emph{multi-class} formulation that predicts a single CWE per CVE and a \emph{multi-label} formulation that allows multiple assignments. Three transformer encoders (BERT Base, SecureBERT, and CySecBERT) are evaluated on three nested label spaces (83, 47, and 25 classes). Multi-class training achieves higher macro-F1 across all settings, although the gap to multi-label narrows from 21 to 2 percentage points as the label space shrinks. Post-hoc threshold optimisation on the multi-label side closes this gap on the 25-class setting. Confusion analysis shows that the dominant misclassification patterns follow the CWE hierarchy and are shared across all three encoders (Pearson $r > 0.92$), which suggests that the error structure is driven more by taxonomy design than by encoder choice. A hierarchy-relaxed evaluation that forgives within-family confusions raises macro-F1 from ${\sim}$81\% to ${\sim}$90\%, indicating that strict metrics understate branch-level classifier quality. CySecBERT achieves the strongest results overall, with statistically significant gains concentrated in the multi-label setting.

Problem

• CVE-to-CWE mapping
• multi-class classification
• multi-label classification
• taxonomy hierarchy
• vulnerability classification

Innovation

• multi-label classification
• CVE-to-CWE mapping
• hierarchical taxonomy
• BERT variants
• threshold optimisation