QLPro: Automated Code Vulnerability Discovery via LLM and Static Code Analysis Integration

2025-06-30

AI Summary

Existing static analysis tools (e.g., CodeQL) suffer from low vulnerability recall in open-source projects due to limited contextual understanding and imprecise vulnerability pattern generalization. Method: This paper proposes QLPro, the first systematic framework integrating large language models (LLMs) with static code analysis—leveraging CodeQL as its underlying engine while employing LLMs to enhance context-aware code comprehension and vulnerability reasoning. A dedicated Java benchmark dataset, JavaTest, is constructed for rigorous evaluation. Contribution/Results: On JavaTest, QLPro detects 41 known vulnerabilities—70.8% more than CodeQL—and discovers six previously unknown vulnerabilities, including two confirmed 0-day vulnerabilities. The framework significantly advances deep, systematic vulnerability discovery in large-scale codebases and establishes a reproducible methodology and empirical benchmark for LLM-augmented static analysis.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Language and Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

We introduce QLPro, a vulnerability detection framework that systematically integrates LLMs and static analysis tools to enable comprehensive vulnerability detection across entire open-source projects.We constructed a new dataset, JavaTest, comprising 10 open-source projects from GitHub with 62 confirmed vulnerabilities. CodeQL, a state-of-the-art static analysis tool, detected only 24 of these vulnerabilities while QLPro detected 41. Furthermore, QLPro discovered 6 previously unknown vulnerabilities, 2 of which have been confirmed as 0-days.

Problem

• Integrating LLMs and static analysis for vulnerability detection
• Improving detection rates compared to standalone static analysis tools
• Discovering previously unknown zero-day vulnerabilities in open-source projects

Innovation

• Integrates LLMs with static code analysis
• Detects more vulnerabilities than CodeQL
• Discovers previously unknown 0-day vulnerabilities