VULSOVER: Vulnerability Detection via LLM-Driven Constraint Solving

2025-08-31

AI Summary

Traditional vulnerability detection methods suffer from either over-reliance on handcrafted rules or hallucination and instability inherent in large language models (LLMs), compromising both accuracy and robustness. This paper proposes an LLM-driven constraint-solving framework that formulates vulnerability detection as a semantics-aware constraint satisfaction problem, synergistically integrating static application security testing (SAST) with LLM-based code understanding: SAST extracts structured, syntactic constraints, while the LLM performs semantic-level reasoning and validation—enabling expert-like collaborative inference. By decoupling structural analysis from semantic interpretation, the approach avoids the semantic blind spots of pure rule-based matching and mitigates the unreliability of end-to-end LLM invocation. Evaluated on the OWASP Benchmark, it achieves 96.29% accuracy, 96.55% F1-score, and 100% recall. Furthermore, it identifies 15 previously unreported high-severity vulnerabilities across real-world GitHub projects.

Technology Category

• Constraint Satisfaction and Optimization: Satisfiability
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Traditional vulnerability detection methods rely heavily on predefined rule matching, which often fails to capture vulnerabilities accurately. With the rise of large language models (LLMs), leveraging their ability to understand code semantics has emerged as a promising direction for achieving more accurate and efficient vulnerability detection. However, current LLM-based approaches face significant challenges: instability in model outputs, limitations in context length, and hallucination. As a result, many existing solutions either use LLMs merely to enrich predefined rule sets, thereby keeping the detection process fundamentally rule-based, or over-rely on them, leading to poor robustness. To address these challenges, we propose a constraint-solving approach powered by LLMs named VULSOLVER. By modeling vulnerability detection as a constraint-solving problem, and by integrating static application security testing (SAST) with the semantic reasoning capabilities of LLMs, our method enables the LLM to act like a professional human security expert. We assess VULSOLVER on the OWASP Benchmark (1,023 labeled samples), achieving 96.29% accuracy, 96.55% F1-score, and 100% recall. Applied to popular GitHub repositories, VULSOLVER also identified 15 previously unknown high-severity vulnerabilities (CVSS 7.5-9.8), demonstrating its effectiveness in real-world security analysis.

Problem

• Improving vulnerability detection accuracy via LLM-driven constraint solving
• Addressing instability and hallucination in LLM-based security analysis
• Integrating SAST with LLM semantics for expert-level vulnerability identification

Innovation

• LLM-driven constraint solving for vulnerability detection
• Integrating SAST with LLM semantic reasoning
• Modeling detection as constraint-solving problem