What Do They Fix? LLM-Aided Categorization of Security Patches for Critical Memory Bugs

2025-09-26

AI Summary

Addressing the challenge of delayed downstream patching and prolonged vulnerability windows due to difficulties in identifying Linux kernel security patches—exacerbated by CVE omissions/delays, silent fixes, and evolving patch standards—this paper proposes DUALLM, a dual-model framework integrating a large language model (LLM) with a fine-tuned lightweight model. DUALLM jointly analyzes commit titles, messages, code diffs, and contextual information to automatically detect fine-grained memory-safety vulnerabilities (e.g., out-of-bounds access, use-after-free) in kernel patches. Evaluated on real-world kernel commits, it achieves 87.4% accuracy and an F1-score of 0.875, significantly outperforming prior approaches. The method identifies 111 high-severity patches, of which 90 are confirmed as true positives via manual validation; two conceptually novel proof-of-concept exploits are successfully constructed from newly discovered vulnerabilities. This advances timely critical patch identification and accelerates vulnerability response in the Linux ecosystem.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Open-source software projects are foundational to modern software ecosystems, with the Linux kernel standing out as a critical exemplar due to its ubiquity and complexity. Although security patches are continuously integrated into the Linux mainline kernel, downstream maintainers often delay their adoption, creating windows of vulnerability. A key reason for this lag is the difficulty in identifying security-critical patches, particularly those addressing exploitable vulnerabilities such as out-of-bounds (OOB) accesses and use-after-free (UAF) bugs. This challenge is exacerbated by intentionally silent bug fixes, incomplete or missing CVE assignments, delays in CVE issuance, and recent changes to the CVE assignment criteria for the Linux kernel. While fine-grained patch classification approaches exist, they exhibit limitations in both coverage and accuracy. In this work, we identify previously unexplored opportunities to significantly improve fine-grained patch classification. Specifically, by leveraging cues from commit titles/messages and diffs alongside appropriate code context, we develop DUALLM, a dual-method pipeline that integrates two approaches based on a Large Language Model (LLM) and a fine-tuned small language model. DUALLM achieves 87.4% accuracy and an F1-score of 0.875, significantly outperforming prior solutions. Notably, DUALLM successfully identified 111 of 5,140 recent Linux kernel patches as addressing OOB or UAF vulnerabilities, with 90 true positives confirmed by manual verification (many do not have clear indications in patch descriptions). Moreover, we constructed proof-of-concepts for two identified bugs (one UAF and one OOB), including one developed to conduct a previously unknown control-flow hijack as further evidence of the correctness of the classification.

Problem

• Classifying security-critical patches for memory bugs in Linux kernel
• Addressing coverage and accuracy limitations in patch classification methods
• Identifying exploitable vulnerabilities like out-of-bounds and use-after-free bugs

Innovation

• LLM-based dual-method pipeline for patch classification
• Leverages commit messages and code context
• Identifies exploitable memory vulnerabilities with high accuracy