RAGuard: A Novel Approach for in-context Safe Retrieval Augmented Generation for LLMs

2025-09-03

AI Summary

To address the insufficient accuracy and safety compliance of large language models (LLMs) in safety-critical offshore wind (OSW) maintenance scenarios, this paper proposes RAGuard—a novel retrieval-augmented generation framework. RAGuard innovatively decouples knowledge retrieval from safety regulation retrieval, introducing independent retrieval budgets and a SafetyClamp hard constraint mechanism. It supports parallel dual-index querying and candidate pool expansion across BM25, dense passage retrieval (DPR), and hybrid retrieval paradigms. Experiments demonstrate that while maintaining technical recall above 60%, safety recall improves dramatically—from near 0% to over 50%. This marks the first demonstration of concurrent technical depth and comprehensive safety coverage in LLM-generated OSW operational guidance. RAGuard thus delivers a verifiable, engineering-grade solution for high-reliability AI decision support in critical infrastructure domains.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Accuracy and safety are paramount in Offshore Wind (OSW) maintenance, yet conventional Large Language Models (LLMs) often fail when confronted with highly specialised or unexpected scenarios. We introduce RAGuard, an enhanced Retrieval-Augmented Generation (RAG) framework that explicitly integrates safety-critical documents alongside technical manuals.By issuing parallel queries to two indices and allocating separate retrieval budgets for knowledge and safety, RAGuard guarantees both technical depth and safety coverage. We further develop a SafetyClamp extension that fetches a larger candidate pool, "hard-clamping" exact slot guarantees to safety. We evaluate across sparse (BM25), dense (Dense Passage Retrieval) and hybrid retrieval paradigms, measuring Technical Recall@K and Safety Recall@K. Both proposed extensions of RAG show an increase in Safety Recall@K from almost 0% in RAG to more than 50% in RAGuard, while maintaining Technical Recall above 60%. These results demonstrate that RAGuard and SafetyClamp have the potential to establish a new standard for integrating safety assurance into LLM-powered decision support in critical maintenance contexts.

Problem

• Ensuring safety in offshore wind maintenance using LLMs
• Addressing specialized scenarios where conventional LLMs fail
• Integrating safety-critical documents with technical information retrieval

Innovation

• Parallel queries to dual indices for knowledge and safety
• SafetyClamp extension with hard-clamping exact slot guarantees
• Hybrid retrieval evaluation across BM25, dense and paradigms