LLM-driven Provenance Forensics for Threat Investigation and Detection

2025-08-29

AI Summary

To address challenges in Advanced Persistent Threat (APT) provenance analysis—including fragmented intelligence extraction, unreliable reasoning, and unverifiable results—this paper proposes PROVSEEK, an LLM-driven, provenance-aware forensic analysis framework. Methodologically, it introduces a role-specialized multi-agent collaboration paradigm that integrates Retrieval-Augmented Generation (RAG), Chain-of-Thought (CoT) reasoning, and a vectorized threat knowledge base to enable context-aware query generation over provenance graphs and interpretable attack behavior inference. Through agent分工 and knowledge-guided hallucination suppression, PROVSEEK produces structured, verifiable forensic summaries. Experiments on the DARPA dataset demonstrate that PROVSEEK improves intelligence extraction accuracy and recall by 34%, and boosts APT detection precision and recall by 22% and 29%, respectively—significantly outperforming existing baselines and state-of-the-art methods.

Technology Category

• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Security and Privacy: Data transparency and provenance
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

We introduce PROVSEEK, an LLM-powered agentic framework for automated provenance-driven forensic analysis and threat intelligence extraction. PROVSEEK employs specialized toolchains to dynamically retrieve relevant context by generating precise, context-aware queries that fuse a vectorized threat report knowledge base with data from system provenance databases. The framework resolves provenance queries, orchestrates multiple role-specific agents to mitigate hallucinations, and synthesizes structured, ground-truth verifiable forensic summaries. By combining agent orchestration with Retrieval-Augmented Generation (RAG) and chain-of-thought (CoT) reasoning, PROVSEEK enables adaptive multi-step analysis that iteratively refines hypotheses, verifies supporting evidence, and produces scalable, interpretable forensic explanations of attack behaviors. By combining provenance data with agentic reasoning, PROVSEEK establishes a new paradigm for grounded agentic forecics to investigate APTs. We conduct a comprehensive evaluation on publicly available DARPA datasets, demonstrating that PROVSEEK outperforms retrieval-based methods for intelligence extraction task, achieving a 34% improvement in contextual precision/recall; and for threat detection task, PROVSEEK achieves 22%/29% higher precision/recall compared to both a baseline agentic AI approach and State-Of-The-Art (SOTA) Provenance-based Intrusion Detection System (PIDS).

Problem

• Automating provenance-driven forensic analysis for threat intelligence extraction
• Resolving provenance queries and mitigating hallucinations in forensic summaries
• Enabling adaptive multi-step analysis to investigate advanced persistent threats

Innovation

• LLM-powered agentic framework for automated forensic analysis
• Combines RAG and chain-of-thought reasoning for multi-step analysis
• Fuses threat report knowledge base with system provenance data