MAVUL: Multi-Agent Vulnerability Detection via Contextual Reasoning and Interactive Refinement

2025-09-30

AI Summary

Existing vulnerability detection methods suffer from insufficient contextual understanding, limitations of single-turn interaction, and biases arising from coarse-grained evaluation. To address these issues, this paper proposes a multi-agent collaborative vulnerability detection framework featuring an innovative dual-role iterative feedback mechanism—comprising a *Vulnerability Analyst* and a *Security Architect*—that jointly performs cross-procedural code comprehension, tool-augmented reasoning, and interactive refinement to achieve context-aware dynamic analysis. Furthermore, we introduce a multidimensional ground-truth evaluation system enabling fine-grained, interpretable, and unbiased vulnerability identification and assessment. Experiments on a pairwise vulnerability dataset demonstrate that our approach achieves over 62% higher accuracy than state-of-the-art multi-agent systems and over 600% improvement over single-agent baselines; notably, performance consistently improves with increasing communication rounds.

Technology Category

• Multiagent Systems: Adversarial Agents
• Computer Vision: Multi-modal Vision
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The widespread adoption of open-source software (OSS) necessitates the mitigation of vulnerability risks. Most vulnerability detection (VD) methods are limited by inadequate contextual understanding, restrictive single-round interactions, and coarse-grained evaluations, resulting in undesired model performance and biased evaluation results. To address these challenges, we propose MAVUL, a novel multi-agent VD system that integrates contextual reasoning and interactive refinement. Specifically, a vulnerability analyst agent is designed to flexibly leverage tool-using capabilities and contextual reasoning to achieve cross-procedural code understanding and effectively mine vulnerability patterns. Through iterative feedback and refined decision-making within cross-role agent interactions, the system achieves reliable reasoning and vulnerability prediction. Furthermore, MAVUL introduces multi-dimensional ground truth information for fine-grained evaluation, thereby enhancing evaluation accuracy and reliability. Extensive experiments conducted on a pairwise vulnerability dataset demonstrate MAVUL's superior performance. Our findings indicate that MAVUL significantly outperforms existing multi-agent systems with over 62% higher pairwise accuracy and single-agent systems with over 600% higher average performance. The system's effectiveness is markedly improved with increased communication rounds between the vulnerability analyst agent and the security architect agent, underscoring the importance of contextual reasoning in tracing vulnerability flows and the crucial feedback role. Additionally, the integrated evaluation agent serves as a critical, unbiased judge, ensuring a more accurate and reliable estimation of the system's real-world applicability by preventing misleading binary comparisons.

Problem

• Detects software vulnerabilities through multi-agent contextual reasoning
• Enhances code understanding via interactive refinement between specialized agents
• Improves evaluation accuracy with multi-dimensional ground truth information

Innovation

• Multi-agent system with contextual reasoning for vulnerability detection
• Iterative feedback and refined decision-making between agents
• Multi-dimensional ground truth for fine-grained evaluation accuracy