CyberBOT: Towards Reliable Cybersecurity Education via Ontology-Grounded Retrieval Augmented Generation

2025-04-01

AI Summary

Large language models (LLMs) often generate inaccurate or unsafe responses when applied to cybersecurity education, posing risks to pedagogical integrity and student safety. Method: This paper proposes a multi-stage Retrieval-Augmented Generation (RAG) question-answering system integrated with a domain-specific cybersecurity ontology. The ontology serves as both a reasoning constraint and a verification layer within the RAG pipeline, enabling semantic consistency checking and security compliance validation. The system synergistically combines course material retrieval, domain-adapted LLM fine-tuning, formal ontology modeling, and rule-driven response verification. Contribution/Results: Deployed in a graduate-level cybersecurity course at Arizona State University involving over 100 students, the system demonstrates significant improvements in answer credibility and instructional relevance. It establishes a reproducible methodology and practical paradigm for deploying trustworthy AI in high-stakes technical education domains.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Application Domains: Security

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Security and Privacy: Large-scale security measurements
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Advancements in large language models (LLMs) have enabled the development of intelligent educational tools that support inquiry-based learning across technical domains. In cybersecurity education, where accuracy and safety are paramount, systems must go beyond surface-level relevance to provide information that is both trustworthy and domain-appropriate. To address this challenge, we introduce CyberBOT, a question-answering chatbot that leverages a retrieval-augmented generation (RAG) pipeline to incorporate contextual information from course-specific materials and validate responses using a domain-specific cybersecurity ontology. The ontology serves as a structured reasoning layer that constrains and verifies LLM-generated answers, reducing the risk of misleading or unsafe guidance. CyberBOT has been deployed in a large graduate-level course at Arizona State University (ASU), where more than one hundred students actively engage with the system through a dedicated web-based platform. Computational evaluations in lab environments highlight the potential capacity of CyberBOT, and a forthcoming field study will evaluate its pedagogical impact. By integrating structured domain reasoning with modern generative capabilities, CyberBOT illustrates a promising direction for developing reliable and curriculum-aligned AI applications in specialized educational contexts.

Problem

• Ensuring accurate and safe cybersecurity education via AI
• Combining retrieval-augmented generation with domain-specific ontology
• Reducing misleading guidance in technical learning tools

Innovation

• Uses retrieval-augmented generation (RAG) pipeline
• Validates responses via cybersecurity ontology
• Integrates structured reasoning with generative AI