Agentic and Generative AI for Open-Source Intelligence and Cyber Investigations: Taxonomy, Evaluation, Challenges, and Future Directions

2026-07-03

AI Summary

This study addresses the limitations of traditional open-source intelligence (OSINT) analysis in handling vast volumes of publicly available data and the inadequacy of existing evaluation frameworks in keeping pace with advances in large language models (LLMs) and agent-based AI. Through a systematic review of 74 studies, it establishes agent AI as a distinct analytical paradigm and introduces a novel taxonomy comprising eleven categories—including LLM foundations, agent architectures, retrieval-augmented generation (RAG), and knowledge graphs—to map current support across the OSINT lifecycle. The analysis reveals robust capabilities in information collection and analysis but significant gaps in verification, reporting, and decision support, with only one study measuring end-to-end hallucination under non-reproducible conditions. The paper proposes a human–AI collaborative deployment architecture for near-term adoption and outlines a ten-point research agenda covering evaluation, benchmarking, robustness, dark web integration, multimodality, and governance.

Abstract

The rapid growth of publicly available digital information has rendered manual open-source intelligence (OSINT) analysis insufficient for modern intelligence, cybersecurity, and cyber investigation. Large language models (LLMs) and agentic AI systems, capable of tool use, multi-step reasoning, and iterative intelligence generation, have emerged as promising solutions, yet evaluation frameworks have not kept pace with reported capabilities. This survey systematically reviews 74 studies and makes four contributions. First, it establishes agentic AI as a distinct analytical category rather than an extension of LLM prompting, organising the literature through an 11-category taxonomy covering LLM foundations, agentic architectures, retrieval-augmented generation (RAG), knowledge graphs, prompt engineering, domain adaptation, evaluation benchmarks, and risk. Second, it identifies the hallucination-validation gap as a corpus-level finding: although hallucination is recognised as a major reliability concern in over twenty studies, end-to-end hallucination is empirically measured in only one OSINT-specific RAG-based system, non-reproducible conditions, while related reasoning and factual-correction studies evaluate general-domain question answering rather than OSINT. Third, it maps existing research to the OSINT lifecycle, showing strong support for collection and analysis but limited coverage of verification, reporting, dissemination, and decision support. Fourth, it derives a ten-point research agenda addressing evaluation, benchmarking, hallucination measurement, adversarial robustness, dark-web coverage, multimodal intelligence, and governance. It concludes that a human-AI co-pilot model, where LLMs assist collection and triage while analysts retain responsibility for verification and decision-making, represents the most defensible near-term deployment architecture.

Problem

• Open-Source Intelligence
• Agentic AI
• Hallucination Validation
• Evaluation Framework
• Cyber Investigations

Innovation

• Agentic AI
• Open-Source Intelligence (OSINT)
• Hallucination-Validation Gap
• Retrieval-Augmented Generation (RAG)
• Human-AI Copilot