Agentic AI for Substance Use Education: Integrating Regulatory and Scientific Knowledge Sources

2026-05-01

AI Summary

Traditional substance use education suffers from limited scalability, insufficient personalization, and outdated information. This work proposes an agent-based AI educational system that uniquely integrates Drug Enforcement Administration regulatory data with dynamically updated scientific literature. By leveraging retrieval-augmented generation (RAG), semantic chunking with vector storage, and real-time PubMed queries, the system establishes a verifiable, context-sensitive, and continuously refreshed health education framework. Evaluated across four key dimensions—factual accuracy, citation quality, contextual coherence, and regulatory compliance—the system achieved average scores ranging from 4.18 to 4.35. Expert assessments demonstrated high inter-rater agreement (Cohen’s κ = 0.78), confirming the system’s effectiveness and reliability.

Abstract

The delivery of traditional substance education has remained problematic due to challenges in scalability, personalization, and the currency of information in a rapidly evolving substance use landscape. While artificial intelligence (AI) offers a promising frontier for enhancing educational delivery, its application in providing real-time, authoritative substance use education remains largely underexplored. We built an agentic-based AI web application that combined Drug Enforcement Administration records with peer-reviewed literature in real-time to provide transparent context-sensitive substance use education. The system uses retrieval-augmented generation with a carefully filtered corpus of 102 documents and dynamic PubMed queries. Document storage was semantically chunked and placed in a vector representation in order to be easily retrieved. We conducted an expert evaluation study in which a panel of five subject matter experts generated 30 domain-specific questions, and two independent raters assessed 90 system interactions (30 primary questions plus two contextual follow-ups each) using a five-point Likert scale across four criteria: factual accuracy, citation quality, contextual coherence, and regulatory appropriateness. Mean ratings ranged from 4.18 to 4.35 across the four criteria (overall category range: 4.05-4.52), with substantial inter-rater agreement (Cohen's kappa = 0.78). These findings suggest that agentic AI architectures integrating authoritative regulatory sources with real-time scientific literature represent a promising direction for scalable, accurate, and verifiable health education delivery, warranting further evaluation through longitudinal user studies.

Problem

• substance use education
• scalability
• personalization
• information currency
• real-time authoritative education

Innovation

• Agentic AI
• Retrieval-Augmented Generation
• Substance Use Education
• Regulatory Knowledge Integration
• Real-time Scientific Literature